Make the root-downloaded images available to non-root users

DRAFT: DOESN'T WORK YET, the files are not getting the xattr set because
of bootc/ostree limitations

# Background

The RHEL-AI bootc image contains the ilab container image pre-pulled.
This image is downloaded using the `root` user into a container image
storage.

# Issue

Non-root users which try to run the `ilab` wrapper which launches a
container using the `ilab` image face a very large pull because as of
today, they're not even using the storage root pulls into as an
additional image store

# Solution

Modify the wrapper such that podman will use the root storage
configuration. This is achieved through the `--storage-opt" "additionalimagestore=/usr/lib/containers/storage"`
flag

# Issue 2

Since the storage is populated by the root user, the storage is
readable/searchable only by root.

# Solution

This change runs `chmod a+rx -R /usr/lib/containers` to make the storage
directory and its contents accessible to non-root users.

However, this has the effect of modifying the permissions of all the
in-container files to be world-readable and world-executable. This is
not ideal. To overcome this, we use `--storage-opt
'overlay.force_mask=shared' --storage-opt
'overlay.mount_program=/usr/bin/fuse-overlayfs'` to modify the storage
config to `force_mask` `shared` (alias for `0755`) and the
`mount_program` to `/usr/bin/fuse-overlayfs`, so that the original
in-image file permissions are recorded in the
`user.containers.override_stat` xattr.

training/ilab-wrapper/ilab

@@ -16,11 +16,15 @@ if [[ "$1" = "shell" ]]; then
 	export PARAMS=()
 fi
 
-PODMAN_COMMAND=("podman" "run" "--rm" "-it"
+PODMAN_COMMAND=(
+	"podman" "run" "--rm" "-it"
 	"--device" "${CONTAINER_DEVICE}"
 	"--security-opt" "label=disable" "--net" "host"
 	"-v" "$HOME:$HOME"
-        "--env" "HOME"
+	"--storage-opt" "overlay.mount_program=/usr/bin/fuse-overlayfs"
+	"--storage-opt" "overlay.force_mask=shared"
+	"--storage-opt" "additionalimagestore=/usr/lib/containers/storage"
+	"--env" "HOME"
 	"--entrypoint" "$ENTRYPOINT"
 	"--env" "HF_TOKEN"
 	"${IMAGE_NAME}")

training/nvidia-bootc/Containerfile

@@ -206,13 +206,16 @@ VOLUME /var/lib/containers
 
 RUN --mount=type=secret,id=${INSTRUCTLAB_IMAGE_PULL_SECRET}/.dockerconfigjson \
     if [ -f "/run/.input/instructlab-nvidia/oci-layout" ]; then \
-         IID=$(podman --root /usr/lib/containers/storage pull oci:/run/.input/instructlab-nvidia) && \
+         IID=$(podman --root /usr/lib/containers/storage --storage-opt 'overlay.force_mask=shared' --storage-opt 'overlay.mount_program=/usr/bin/fuse-overlayfs' pull oci:/run/.input/instructlab-nvidia) && \
          podman --root /usr/lib/containers/storage image tag ${IID} ${INSTRUCTLAB_IMAGE}; \
     elif [ -f "/run/secrets/${INSTRUCTLAB_IMAGE_PULL_SECRET}/.dockerconfigjson" ]; then \
-         IID=$(sudo podman --root /usr/lib/containers/storage pull --authfile /run/secrets/${INSTRUCTLAB_IMAGE_PULL_SECRET}/.dockerconfigjson ${INSTRUCTLAB_IMAGE}); \
+         IID=$(sudo podman --root /usr/lib/containers/storage --storage-opt 'overlay.force_mask=shared' --storage-opt 'overlay.mount_program=/usr/bin/fuse-overlayfs' pull --authfile /run/secrets/${INSTRUCTLAB_IMAGE_PULL_SECRET}/.dockerconfigjson ${INSTRUCTLAB_IMAGE}); \
     else \
-         IID=$(sudo podman --root /usr/lib/containers/storage pull ${INSTRUCTLAB_IMAGE}); \
-    fi
+         IID=$(sudo podman --root /usr/lib/containers/storage --storage-opt 'overlay.force_mask=shared' --storage-opt 'overlay.mount_program=/usr/bin/fuse-overlayfs' pull ${INSTRUCTLAB_IMAGE}); \
+    fi \
+    && chmod a+rx -R /usr/lib/containers
+
 RUN podman system reset --force 2>/dev/null
 
 LABEL image_version_id="${IMAGE_VERSION_ID}"
+