BUG: tagging is broken for multistage builds

[h-vetinari]

[Now that I have access to the nightly ppa's (cf. containers/podman#2250), I'm starting to file some issues against podman - and since the issue template there says that podman build issues should be filed for buildah, I'm starting here.]

Assume I have a toy dockerfile, say podman_test.dockrf:

FROM docker.io/library/ubuntu:bionic

RUN apt-get update && apt-get install curl -y

CMD ["bash"]

Then sudo podman build -t podman_test -f podman_test.dockrf . produces

So far so good. If I change the dockerfile slightly:

-FROM docker.io/library/ubuntu:bionic
+FROM docker.io/library/ubuntu:bionic as base

 RUN apt-get update && apt-get install curl -y

 CMD ["bash"]

then I get an untagged image (even though it successfully uses the cache):

Same for sudo podman build -t podman_test --target=base -f podman_test.dockrf .

This is quite basic functionality IMO - without tagging, many automation tasks are not possible.

[TomSweeneyRedHat]

@h-vetinari thanks for the issue. This is definitely not acting correctly as you surmised. I think the <none><none> is ok for image 21fe, but what I think should be happening is we should have another image there named podman_test with an id of d6ca... , the last line from the build output there.

There appears to be something going on with the --layers option that's not letting that complete successfuly, if I add --layers=false to the podman build command, then I see that image.

I'll have to dig more in the next day or too, and may need to check in with @nalind

[h-vetinari]

Thanks for the response!

@TomSweeneyRedHat: I think the <none> <none> is ok for image 21fe, but what I think should be happening is we should have another image there named podman_test with an id of d6ca... , the last line from the build output there.

Ideally (IMO) both dockerfiles should create exactly the same image, so that 21fe... is not untagged by the build, but stays the same. Builds not being idempotent is another issue I'm encountering (pending further tests).

[nalind]

I think #1473 will fix this.

[h-vetinari]

@nalind @TomSweeneyRedHat
I just tested the newest nightly after I saw that #1473 had been merged - unfortunately, the issue persists (assuming http://ppa.launchpad.net/projectatomic/ppa/ubuntu is up-to-date).

[TomSweeneyRedHat]

@h-vetinari I'm not sure that's up to date. @lsm5 can you confirm or deny please?

[lsm5]

@h-vetinari what version of ubuntu are you on? I see that 16.04 hasn't been updated in a long time. I'll fix that now. 18.04 was last built on 5th april. Travis auto-builds break often for buildah quite likely due to .travis.yml changes.

[h-vetinari]

@lsm5
I'm on Ubuntu 18.04. Is the ppa-update not scheduled regularly (e.g. daily)? Can you say if 29a6c81 was already part of the last build?

@nalind @TomSweeneyRedHat
Seems that the last build I tested above (and which fails the scenario laid out in the OP) includes at least #1473, but possibly not #1487.

[h-vetinari]

@nalind @TomSweeneyRedHat @rhatdan

Just reran the test again, and it still fails, for both of:

sudo podman build -t podman_test -f podman_test.dockrf .
sudo podman build -t podman_test --target=base -f podman_test.dockrf .

Note that this time, apt-get install podman buildah picked up that the ppa had a newer version than yesterday, so it definitely includes #1473 and #1487.

Not sure if #1497 will have any bearing on this, but in any case, it would be very helpful to have this fixed.

[lsm5]

On Mon, Apr 08, 2019 at 01:32:52PM -0700, h-vetinari wrote: @lsm5 I'm on Ubuntu 18.04. Is the ppa-update not scheduled regularly (e.g. daily)? Can you say if 29a6c81 was already part of the last build?

The latest build for 18.04 includes all commits up to 610eb7a so I'll say yes. RE: ppa update, it happens daily on my fork. But things tend to break often cause rebases aren't always clean, and the debian build process makes it super painful to upstream. I have a TODO for snap builds which should be a lot easier to upstream. Soon as I can find the time.

…

@nalind @TomSweeneyRedHat Seems that the last build I tested above (and which fails the scenario laid out in the OP) includes at least #1473, but possibly not #1487. -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #1456 (comment)

-- Lokesh IRC, GitHub: lsm5 GPG: 0xC7C3A0DD https://keybase.io/lsm5

[nalind]

[Now that I have access to the nightly ppa's (cf. containers/libpod#2250), I'm starting to file some issues against podman - and since the issue template there says that podman build issues should be filed for buildah, I'm starting here.]

Assume I have a toy dockerfile, say podman_test.dockrf:

FROM docker.io/library/ubuntu:bionic

RUN apt-get update && apt-get install curl -y

CMD ["bash"]

Then sudo podman build -t podman_test -f podman_test.dockrf . produces

So far so good. If I change the dockerfile slightly:

-FROM docker.io/library/ubuntu:bionic
+FROM docker.io/library/ubuntu:bionic as base

 RUN apt-get update && apt-get install curl -y

 CMD ["bash"]

then I get an untagged image (even though it successfully uses the cache):

Same for sudo podman build -t podman_test --target=base -f podman_test.dockrf .

This is quite basic functionality IMO - without tagging, many automation tasks are not possible.

I think what you're after would conventionally be accomplished by building only through that stage with the --target flag, and by using the -t flag to specify a tag to apply to the final image.

The name specified in a FROM ... AS ... instruction is not treated as a tag for the image built at that stage, as its main use is to avoid requiring that subsequent COPY --from= instructions specify stages by number, which can get hard to maintain if you find you need to add a stage to the middle of a multi-stage Dockerfile.

[h-vetinari]

@h-vetinari: [...]
Same for sudo podman build -t podman_test --target=base -f podman_test.dockrf .
[...]

@nalind: I think what you're after would conventionally be accomplished by building only through that stage with the --target flag, and by using the -t flag to specify a tag to apply to the final image.

This is exactly what is not working, and what this issue is about, see self-quote from OP. Neither does it work to build the full image (no --target) - the final image currently ignores the -t tag.

[nalind]

Aargh, I missed that. This appears to work correctly with buildah at least as of version ffd8d6a, though, so I expect we're looking at containers/podman#2931 to incorporate those changes into podman.

[h-vetinari]

@nalind
I've been trying to monitor the PRs and regularly tested with the most current nightly builds for podman/buildah. Seems that this specific commit hasn't made it to the PPA yet, so I can't verify. But thanks for fixing! :)

Yeah, would definitely need to update the version vendored by podman, as I'm planning to use this through podman.

[h-vetinari]

@nalind
Never mind, I hadn't completely thought through the effects of the vendoring...

Using the latest PPA, it works correctly to do either of:

• sudo buildah bud -t podman_test -f podman_test.dockrf .
• sudo buildah bud -t podman_test --target=base -f podman_test.dockrf .

[h-vetinari]

Thanks!

[h-vetinari]

Just checked with the new PPA after containers/podman#2931, and now it works with podman build too! :)