ARG does not get expanded in a COPY --from directive

[aoikonomopoulos]

Description
This seems pretty similar to #2192; With the Dockerfile

ARG base
FROM ${base}
ARG toolchain_name
ARG destination_path

COPY --from=${toolchain_name} / ${destination_path}

buildah bud command fails with:

$ buildah bud --build-arg base=existing_image_1 --build-arg toolchain_name=existing_image_2 --build-arg destination_path=/blah
STEP 1: FROM localhost/yeoldeworkername1-no-toolchain
STEP 2: ARG toolchain_name
STEP 3: ARG destination_path
STEP 4: COPY --from=${toolchain_name} / ${destination_path}
COPY --from=${toolchain_name}: no stage or image found with that name

whereas it works when using docker build.

Steps to reproduce the issue:

1. Create the Dockerfile above
2. Run the command above, substituting (arbitrary) existing image names

Describe the results you received:

toolchain_name not expanded.

Describe the results you expected:

toolchain_name expanded to existing_image_2.

Output of rpm -q buildah

buildah-1.15.0-1.fc32.x86_64

Output of buildah version:

Tried both with the fedora package

Version:         1.15.0
Go Version:      go1.14.3
Image Spec:      1.0.1-dev
Runtime Spec:    1.0.2-dev
CNI Spec:        0.4.0
libcni Version:  
image Version:   5.5.1
Git Commit:      
Built:           Thu Jan  1 01:00:00 1970
OS/Arch:         linux/amd64

and with the latest git (as of a few minutes ago):

Version:         1.16.0-dev
Go Version:      go1.14.3
Image Spec:      1.0.1-dev
Runtime Spec:    1.0.2-dev
CNI Spec:        0.4.0
libcni Version:  v0.7.2-0.20190904153231-83439463f784
image Version:   5.5.1
Git Commit:      03d782c
Built:           Tue Jul 28 17:11:00 2020
OS/Arch:         linux/amd64

Output of cat /etc/*release:

Fedora release 32 (Thirty Two)
NAME=Fedora
VERSION="32 (Thirty Two)"
ID=fedora
VERSION_ID=32
VERSION_CODENAME=""
PLATFORM_ID="platform:f32"
PRETTY_NAME="Fedora 32 (Thirty Two)"
ANSI_COLOR="0;34"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:32"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f32/system-administrators-guide/"
SUPPORT_URL="https://fedoraproject.org/wiki/Communicating_and_getting_help"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=32
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=32
PRIVACY_POLICY_URL="https://fedoraproject.org/wiki/Legal:PrivacyPolicy"
Fedora release 32 (Thirty Two)
Fedora release 32 (Thirty Two)

Output of uname -a:

Linux <elided> 4.19.107-1.pvops.qubes.x86_64 #1 SMP Sat Feb 29 16:04:42 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

[aoikonomopoulos]

AFAICT, the relevant code in imagebuilder (updated in the commit resolving #2192, i.e. COPY --chown= handling) is not executed at all in this testcase. Instead, it looks like we end up in

buildah/imagebuildah/executor.go

Line 491 in 03d782c

case "ADD", "COPY":

which has a TODO about doing ARG expansion (this goes back to commit 391a5be). Not sure what the proper fix is here.

CC: @nalind who may know more about this as they added the TODOs

[TomSweeneyRedHat]

@aoikonomopoulos it is indeed sort of related to #2192, but the issue is at this line:

[TomSweeneyRedHat]

https://github.com/containers/buildah/blob/master/imagebuildah/stage_executor.go#L889

(Not sure how I managed to end the prior comment so quickly). At this line we have the variable name ${toolchain_name} that has yet to be resolved. We do that resolution down in the ImageBuilder code. However, at this line in the stage_executor we're telling the code to look for the stage called ${toolchain_name} rather than the resolved value existing_image_2 as we should. If I hard code the image name instead of using the variable, it works fine.

Now I need to figure out how to resolve that here rather than ImageBuilder, an excercise that I'll leave until tomorrow when I'm not fighting with my eyelids quite as much.

[TomSweeneyRedHat]

Fixed with #2522