Allow --build-arg to override FROM in Dockerfile

[nnachefski]

I would like to be able to over override the FROM image in the Dockerfile from the command line

Example:

# buildah bud --build-arg=FROM=ubi8-custom --format=oci --tls-verify=false --layers -f Dockerfile.rhel8 -t image-registry.openshift-image-registry.svc:5000/custom-images/s2i-custom-core .
STEP 1: FROM registry.access.redhat.com/ubi8:latest
...

[rhatdan]

Interested in opening a PR to make this happen?

[nnachefski]

negative. My Golang is.... nonexistent. I saw that Kaniko does indeed have this functionality. I'm going to give the kaniko ClusterTask a shot.

[rhatdan]

No problem, we will take a look at it.

[jamescassell]

You can already do this:
test.Dockerfile:

ARG FROM=registry.access.redhat.com/ubi8
FROM $FROM
ARG FROM
ENV FROM=$FROM

to swap out the FROM, build it like podman build --build-arg=FROM=registry.fedoraproject.org/fedora -f from.Dockerfile

[rhatdan]

Also we have a preprocessor built into Buildah and Podman build. Basically you can add

if

then

endif

man buildah bud
...
      Dockerfiles  ending  with  a  ".in"  suffix  will  be preprocessed via CPP(1).  This can be useful to decompose Dockerfiles into several reusable parts that can be used via CPP's
       #include directive.  Notice, a Dockerfile.in file can still be used by other tools when manually preprocessing them via cpp -E.

[rhatdan]

@TomSweeneyRedHat Did we ever blog on this feature?

[rhatdan]

If not this entire issue would make for a nice blog.

[TomSweeneyRedHat]

@rhatdan, I don't believe we did. I'll add it to the list.

[nnachefski]

You can already do this:
test.Dockerfile:

ARG FROM=registry.access.redhat.com/ubi8
FROM $FROM
ARG FROM
ENV FROM=$FROM

to swap out the FROM, build it like podman build --build-arg=FROM=registry.fedoraproject.org/fedora -f from.Dockerfile

Thanks for the help folks. This is good news, except i dont have access to change the Dockerfile. Basically, i'm inserting a custom layer on top of ubi8 image, and then using tekton to build a custom s2i chain on top of the customized base. That being said, i wont be able to change your Dockerfile for core/base/python36/node10 etc.... To do this, i would have to fork it and that is not an option as i dont want to maintain a separate code base for each runtime just to insert some ARG into the Dockerfile. I request that this be option be like Kanikos where you dont have to edit the Dockerfile to include the FROM arg before the FROM statement. I'd like it better if the only thing i had to do is patch the ClusterTask to include an extra arg (--build-arg).

[nnachefski]

It's also worth noting that i've been doing this custom s2i chain builds with basic Openshift build objects for awhile now. 'oc new-build' has this functionality as i'm able to override the FROM image with a '-i' in the oc CLI. I want to port this work over to tekton for a customer and that's why i'm doing this now.

[rhatdan]

Ok I am fine with this change.

[rhatdan]

Getting a PR would make it faster.

[rhatdan]

@ashley-cui PTAL

[TomSweeneyRedHat]

Actually I think this is tied in with the one that I'm working on now. I think the fix for #2496 will fit here too. Let me steal this back.

[nnachefski]

Hey Tom,

Would love to help you test this. Where can i get the latest buildah image from so i can see if this works now?

[TomSweeneyRedHat]

Nick, there's no ready made rpm at the moment. So there are two choices, A) you could clone the repo and build from upstream, or B) you could use Podman and run the quay.io/bulidah/upstream:latest image. I just updated that container image on quay.io a little bit ago, so it should have the latest/greatest there.

[nnachefski]

Hey Tom,

I dont think quay.io/bulidah/upstream:latest is a valid image. Or at least, not one that i have privs too see.

EDIT
I found it at quay.io/buildah/upstream:master

[nnachefski]

I'm setting this in my buildah ClusterTask:

--build-arg=FROM=$(resources.inputs.image.url)'

And the image URL is being populated during the Pipeline Run. But, buildah is ignoring the FROM build-arg and using the FROM in the Dockerfile. Any pointers on how to use this properly?

I'm using this buildah image 'quay.io/buildah/upstream:master'

Output from PipelineRun

STEP-CREATE-DIR-IMAGE-ZLPX4

STEP-GIT-SOURCE-S2I-BASE-GIT-FFJVQ

{"level":"info","ts":1598568843.9707274,"caller":"git/git.go:105","msg":"Successfully cloned https://github.com/sclorg/s2i-base-container.git @ master in path /workspace/source"}
{"level":"warn","ts":1598568843.9708374,"caller":"git/git.go:152","msg":"Unexpected error: creating symlink: symlink /tekton/home/.ssh /root/.ssh: file exists"}
{"level":"info","ts":1598568845.1111348,"caller":"git/git.go:133","msg":"Successfully initialized and updated submodules in path /workspace/source"}
STEP-DEBUG

building with ARGS: 'FOO=BAR FROM=image-registry.openshift-image-registry.svc:5000/openshift/ubi8-custom:latest'
STEP-BUILD

STEP 1: FROM registry.access.redhat.com/ubi8:latest
...
...
[Warning] one or more build args were not consumed: [FOO FROM]

[TomSweeneyRedHat]

@nnachefski for this latest error, were you using the same Dockerfile that you included in an earlier comment in this thread?

We've had some issues with ARG handling in Containerfiles when an equal sign was in the mix. As a real quick try, can you drop the first equal sign?

i.e. --build-arg FROM=$(resources.inputs.image.url)'

and try that.

It's way past open eyelid time here, I'll check back in the morning.

[rhatdan]

@nnachefski @TomSweeneyRedHat What is going on with this one?

[nnachefski]

this still doesnt work.

[nnachefski]

Using image quay.io/buildah/upstream:master

STEP-DEBUG
building with ARGS: 'FOO=BAR FROM=ubi8:latest'

STEP-BUILD
STEP 1: FROM ubi8:latest
Getting image source signatures
Copying blob sha256:c4d668e229cd131e0a8e4f8218dca628d9cf9697572875e355fe4b247b6aa9f0
Copying blob sha256:ec1681b6a383e4ecedbeddd5abc596f3de835aed6db39a735f62395c8edbff30
Copying config sha256:ecbc6f53bba0d1923ca9e92b3f747da8353a070fccbae93625bd8b47dbee772e
Writing manifest to image destination
Storing signatures
STEP 2: COPY testing.txt /etc/
STEP 3: COMMIT image-registry.openshift-image-registry.svc:5000/openshift/ubi8-custom:latest
--> fd65e042adf
[Warning] one or more build args were not consumed: [FOO FROM]

[umohnani8]

Looks like this was fixed in #2823. Can this be closed now @TomSweeneyRedHat ?