dockerfile: add support for named contexts

[tonistiigi]

Related moby/moby#37129
Related docker/buildx#447 (comment)

Stages and implicit stages from image names can be
redefined with build options.

This enables using more that one source directory
and reusing results from other builds. This can also
be used to use a local image from other build without
including a registry.

Contexts need to be defined as context:name= frontend
options. The value can be image, git repository,
URL, local directory, or a frontend input.

E.g.

FROM alpine AS base
..
..

buildctl build --frontend=dockerfile.v0 --local context=. --local dockerfile=. --opt context:alpine=docker-image://alpine:3.15

buildctl build --frontend=dockerfile.v0 --local context=. --local dockerfile=. --local base=path/to/base --opt context:base=local:base

Signed-off-by: Tonis Tiigi [email protected]

[tonistiigi]

Added additional test for series of multi-platform builds passing result from first to the next one.

[crazy-max]

Looks great! I have this use case where loaded a local image would be helpful instead of using a local registry.

[rittneje]

@tonistiigi This feature seems really useful. It would be nice to add it to the documentation. Based upon your changeset it seems your options are:

1. docker-image://<image name> to reference an image in a registry
2. git://<git repo url>, http://<git repo url>, https://<git repo url> to reference a Git repository
3. local:<path> to reference a local directory
4. input:<key> to reference some previously built image; requires --opt input-metadata:<key>=<json>, where <json> is, I assume, obtained by reading the output from --metadata-file

[tonistiigi]

https://github.com/docker/buildx/blob/master/docs/reference/buildx_build.md#-additional-build-contexts---build-context

input: can't be really used with buildctl. It's only via API. For example used in https://github.com/docker/buildx/blob/master/docs/reference/buildx_bake.md#using-a-result-of-one-target-as-a-base-image-in-another-target

[rittneje]

Is the example you linked correct? It just has project=path/to/project/source without the local: prefix. Is that only needed for buildctl?

Why can't input be used with buildctl? It would be nice for when we are building a bunch of images off the same base image, and don't want to waste time with the cache or passing build args that the sub-stages don't need.

In any case, I would not generally look to the buildx documentation for buildctl, so it would be nice if it were documented here too.

[tonistiigi]

Is the example you linked correct? It just has project=path/to/project/source without the local: prefix. Is that only needed for buildctl?

Yes. Buildx has a new flag for this. Buildctl has generic --opt that requires prefix and another flag to load the local.

Why can't input be used with buildctl?

Because there is nothing that keeps the result allocated when your first build completes. Pipelines of multiple builds are out of scope for buildctl which is a test client.

In any case, I would not generally look to the buildx documentation for buildctl,

This is a Dockerfile frontend feature. Other frontends can of course use the same keys to also connect to --build-context flag directly. If a frontend looked for the existence of these keys, a feature like this worked on any old version of buildctl.

[rittneje]

I'm confused. To be honest, right now the various --opt flags for buildctl are poorly documented. The readme has a few (unexplained) examples, but there is no authoritative list that I can find. Even if I did think to reference the buildx docs, I certainly wouldn't guess that --build-context becomes --opt context, with an extra quirk for local. I'm going to file a separate issue to improve the buildctl docs on this point.

I'm also not sure what you mean by buildctl being a "test client". The docs never refer to it as such.