multi-stage build fails with unmarshalling error when using file descriptor as authfile

[kkujala]

Description

When I try to build a multi-stage Dockerfile with a file descriptor from bash process substitution, then it fails with unmarshalling error on the second image fetching. The first image is fetched successfully and the second image fetching encounters the error.

In order to hide the authentication credentials from process listings or from filesystem, then creating a temporary file descriptor with bash process substitution is one way of achieving it securely. I.e. using --creds option would reveal the secrets to the host system when running ps. And using an actual file with --authfile would reveal the secrets likewise in the filesystem.

Steps to reproduce the issue:

1. Build the following dockerfile with buildah

$ cat Dockerfile 
FROM docker.io/centos:centos8 as build
FROM myrepository.example/image:tag

The following command should be used for building

buildah bud --authfile=<(echo "{ \"auths\": { \"myrepository.example\": { \"auth\": \"$(echo 'username:password' | base64 --wrap=0)\" } } }") --file Dockerfile --tag myimage .

2. Observe the first image centos8 being fetched correctly

3. Observe the second image myrepository.example/image:tag fetching leading to unmarshalling error.

Describe the results you received:

$ buildah bud --authfile=<(echo "{ \"auths\": { \"myrepository.example\": { \"auth\": \"$(echo 'foo:bar' | base64 --wrap=0)\" } } }") --file Dockerfile --tag myimage .
STEP 1: FROM docker.io/centos:centos8 AS build
Getting image source signatures
Copying blob 7a0437f04f83 [--------------------------------------] 0.0b / 0.0b
Copying config 300e315adb done  
Writing manifest to image destination
Storing signatures
--> 300e315adb2
STEP 2: FROM myrepository.example/image:tag
error creating build container: Error initializing source docker://myrepository.example/image:tag: error getting username and password: error reading JSON file "/dev/fd/63": error unmarshaling JSON at "/dev/fd/63": unexpected end of JSON input
ERRO exit status 125                              

It should be noted that the myrepository.example is intentionally invalid as per RFC 2606, as it is enough to demonstrate this issue. Also the authfile content is not valid but enough for demonstration. One can replace the both of the images with valid images from a repository that requires authentication, and use the correct secrets in the bash substitution. In such case the first image fetching works and the second one still fails with unmarshalling error.

Describe the results you expected:
I'm expecting that the multistage build proceed to fetching the second image with the credentials as given via --authfile using file descriptor from bash process substitution.

Output of rpm -q buildah or apt list buildah:

$ pacman -Qv buildah
Root      : /
Conf File : /etc/pacman.conf
DB Path   : /var/lib/pacman/
Cache Dirs: /var/cache/pacman/pkg/  
Hook Dirs : /usr/share/libalpm/hooks/  /etc/pacman.d/hooks/  
Lock File : /var/lib/pacman/db.lck
Log File  : /var/log/pacman.log
GPG Dir   : /etc/pacman.d/gnupg/
Targets   : buildah
buildah 1.19.7-1

Output of buildah version:

$ buildah version
Version:         1.19.7
Go Version:      go1.16
Image Spec:      1.0.1-dev
Runtime Spec:    1.0.2-dev
CNI Spec:        0.4.0
libcni Version:  v0.8.1
image Version:   5.10.2
Git Commit:      0a1ca424
Built:           Sat Mar  6 00:08:33 2021
OS/Arch:         linux/amd64

Output of podman version if reporting a podman build issue:

$ podman version
Version:      3.0.1
API Version:  3.0.0
Go Version:   go1.16
Git Commit:   c640670e85c4aaaff92741691d6a854a90229d8d
Built:        Sun Feb 21 17:29:46 2021
OS/Arch:      linux/amd64

Output of cat /etc/*release:

$ cat /etc/*release
NAME="Arch Linux"
PRETTY_NAME="Arch Linux"
ID=arch
BUILD_ID=rolling
ANSI_COLOR="38;2;23;147;209"
HOME_URL="https://www.archlinux.org/"
DOCUMENTATION_URL="https://wiki.archlinux.org/"
SUPPORT_URL="https://bbs.archlinux.org/"
BUG_REPORT_URL="https://bugs.archlinux.org/"
LOGO=archlinux

Output of uname -a:

$ uname -a
Linux redacted 5.11.4-arch1-1 #1 SMP PREEMPT Sun, 07 Mar 2021 18:00:49 +0000 x86_64 GNU/Linux

Output of cat /etc/containers/storage.conf:

$ cat /etc/containers/storage.conf | sed -n '/^[^#]/p'
[storage]
driver = "fuse-overlayfs"
runroot = "/run/containers/storage"
graphroot = "/var/lib/containers/storage"
[storage.options]
additionalimagestores = [
]
[storage.options.overlay]
mountopt = "nodev"
[storage.options.thinpool]

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[kkujala]

Would there be any information I could provide that would be helpful?

[rhatdan]

$ cat /tmp/Dockerfile 
from centos as build
from quay.io/rhatdan/myimage
$ buildah bud /tmp
STEP 1: FROM centos AS build
--> 300e315adb2
STEP 2: FROM quay.io/rhatdan/myimage
error creating build container: Error initializing source docker://quay.io/rhatdan/myimage:latest: Error reading manifest latest in quay.io/rhatdan/myimage: unauthorized: access to the requested resource is not authorized
ERRO exit status 125                             
$ buildah bud --authfile=/tmp/login /tmp
STEP 1: FROM centos AS build
--> 300e315adb2
STEP 2: FROM quay.io/rhatdan/myimage
Getting image source signatures
Copying blob 9f3f5280230f done  
Copying blob b3227b79d9b6 done  
Writing manifest to image destination
Storing signatures
STEP 3: COMMIT
--> e4ff4fece70
e4ff4fece70d1d2f0db8e754be03bd1b5ffd5961ba5883fed98d61d36998d4e6

This works for me, and I missing something?

[kkujala]

/tmp/login is an actual file in the filesystem, i.e. it can be accessed multiple times.

Whereas --authfile=<(echo "{ "auths": { "myrepository.example": { "auth": "$(echo 'foo:bar' | base64 --wrap=0)" } } }") is a file descriptor which can be accessed only once. In the second time it yields nothing leading to the unmarshalling error.

Using process substitution prevents the exposure of secrets via filesystem, which is why I'm interested in this.

I suppose one solution would be for buildah to cache the secrets once read and use the cached secrets in multistage builds.

[rhatdan]

Would buildah secrets help with this?

[kkujala]

I suppose you mean "buildah bud --secret=...". I think it is not possible to pass the authentication information for external repository in Dockerfile FROM via buildah secret.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[kkujala]

One solution could be that the authfile content would be read into memory and used from there during the build. There would be less file operations as well.

[rhatdan]

Interested in opening a PR to make this happen?

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@flouthoc PTAL

[flouthoc]

@kkujala thanks for sharing reproducer. We can defiantly write this to memory or a temp file on first pass if input is a file descriptor. Picking this up

[flouthoc]

@kkujala following PR should close this issue #3498