Docker references with both a tag and digest are currently not supported

[devurandom]

Description

Buildah 1.7.1 does not support referencing an image (or pulling it from a registry) with both a tag and a digest. I could not find a feature request about this, so I create this one.

Steps to reproduce the issue:

$ buildah from docker://docker.io/library/clojure:openjdk-8-lein-2.9.1-alpine@sha256:d5454be246358c9cd683a06108ccdecc8b31a0341e45973af480bef708e8cf1e
while pulling "docker://docker.io/library/clojure:openjdk-8-lein-2.9.1-alpine@sha256:d5454be246358c9cd683a06108ccdecc8b31a0341e45973af480bef708e8cf1e" as "docker.io/library/clojure:openjdk-8-lein-2.9.1-alpine@sha256:d5454be246358c9cd683a06108ccdecc8b31a0341e45973af480bef708e8cf1e": error parsing attempted image name "docker://docker.io/library/clojure:openjdk-8-lein-2.9.1-alpine@sha256:d5454be246358c9cd683a06108ccdecc8b31a0341e45973af480bef708e8cf1e": Docker references with both a tag and digest are currently not supported

[TomSweeneyRedHat]

@devurandom Thanks for the issue.

@mtrmac is this anything you forsee changing or should we look into propping up the documentation somehow?

[mtrmac]

We’ve seen this brought up in a one other place WRT podman pull, and Docker does accept that syntax; so, yes, perhaps we would accept this syntax — but only in the compatibility / convenience syntax that does not start with a transport specification (buildah from docker.io/…), as handled in Podman’s pullGoalFromPossiblyUnqualifiedName.

(We’ve been tentatively discussing sharing the pull code between buildah/libpod; this should probably happen in there.)

I don’t expect this syntax to be supported for the docker:// transport directly, primarily because the underlying protocol does not allow specifying both at the same time, so its semantics is ambiguous about users’ intent, and the historical practice is problematic:

• Earlier versions of Docker, including 1.13.1, interpret this as “ask for the image with the specified tag, then fail if the manifest does not match the specified digest”. That’s fairly brittle but ultimately plausible (where the tag ~documents the intent and the digest ensure immutability and prevents moving the tag maliciously).
• Later versions of Docker interpret this as “ask for the image with the specified digest, ignore the tag completely”, which is, in my view, mostly pointless and can be actively misleading. If the tag is ignored in the code, the users’ intent can just as well be noted in a comment after the command instead of inside the image specification, and that would be more accurate WRT what is actually happening.

It may turn out to be necessary to support the :tag@digest syntax for compatibility, but especially when the semantics of that syntax goes over time from “risky but plausible” to “actively confusing”, I don’t think it makes sense to support the “actively confusing” semantics in the new docker:// syntax — and if we are aiming for compatibility, using any but the latest syntax would not be useful either.

[devurandom]

Earlier versions of Docker, including 1.13.1, interpret this as “ask for the image with the specified tag, then fail if the manifest does not match the specified digest”. That’s fairly brittle but ultimately plausible (where the tag ~documents the intent and the digest ensure immutability and prevents moving the tag maliciously).

That is what I expected to happen.

[rhatdan]

How critical is this for you? Why not just pull by digest?

[devurandom]

How critical is this for you? Why not just pull by digest?

I wouldn't say it is critical, but for documentary purposes it is good to have the tag alongside the shasum. I can of course also achieve that by adding a comment (if the file is not JSON, which luckily is seldom these days), which is what I am doing now.

[rhatdan]

Closing this as I don't see anyone working on it.

[chanseokoh]

FYI, Tekton uses both a tag and a digest.

[chanseokoh]

FYI, crio-o fixed this issue and will support using tag + digest: cri-o/cri-o#2351

[riconnon]

Throwing my oar in...
I use the repo:tag@digest syntax to represent a developer-friendly vs. machine-interpretted version of the same thing, somewhat akin to a modern language package manager (package.json/yarn.lock, cargo.toml/cargo.lock)
We use supporting tooling (Renovate: https://docs.renovatebot.com/) which ensures all images referenced by tag have a digest to ensure reproducible builds, and automatically performs CI then auto-merges digest updates which ensures we're aware of breakage when the base images we're using get rebuilt.

The lack of support for this is likely deal breaking for me on the whole containers ecosystem.

[TomSweeneyRedHat]

@riconnon would you mind opening a new issue about what you'd like to see exactly? I'm not entirely clear and would like a dedicated space to discuss/consider it. Thanks!

[rarkins]

@TomSweeneyRedHat I have opened #3331