From 3a32318eb6998073ff70cf5cd32022da618fe780 Mon Sep 17 00:00:00 2001 From: David Karlsson <35727626+dvdksn@users.noreply.github.com> Date: Sat, 24 Feb 2024 13:53:30 +0100 Subject: [PATCH] docs: restructure flags/options for add, copy, run Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com> --- frontend/dockerfile/docs/reference.md | 306 ++++++++++++++------------ 1 file changed, 160 insertions(+), 146 deletions(-) diff --git a/frontend/dockerfile/docs/reference.md b/frontend/dockerfile/docs/reference.md index 48ce59439384e..4a3aa1537a549 100644 --- a/frontend/dockerfile/docs/reference.md +++ b/frontend/dockerfile/docs/reference.md @@ -644,11 +644,11 @@ guide](https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practi The cache for `RUN` instructions can be invalidated by [`ADD`](#add) and [`COPY`](#copy) instructions. -## RUN --mount +### RUN --mount -> **Note** -> -> Added in [`docker/dockerfile:1.2`](#syntax) +```dockerfile +RUN --mount=[type=][,option=[,option=]...] +``` `RUN --mount` allows you to create filesystem mounts that the build can access. This can be used to: @@ -657,9 +657,7 @@ This can be used to: - Access build secrets or ssh-agent sockets - Use a persistent package management cache to speed up your build -Syntax: `--mount=[type=][,option=[,option=]...]` - -### Mount types +The supported mount types are: | Type | Description | | ---------------------------------------- | --------------------------------------------------------------------------------------------------------- | @@ -804,18 +802,16 @@ $ docker buildx build --ssh default=$SSH_AUTH_SOCK . You can also specify a path to `*.pem` file on the host directly instead of `$SSH_AUTH_SOCK`. However, pem files with passphrases are not supported. -## RUN --network +### RUN --network -> **Note** -> -> Added in [`docker/dockerfile:1.1`](#syntax) +```dockerfile +RUN --network= +``` `RUN --network` allows control over which networking environment the command is run in. -Syntax: `--network=` - -### Network types +The supported network types are: | Type | Description | | -------------------------------------------- | -------------------------------------- | @@ -858,15 +854,18 @@ The command is run in the host's network environment (similar to > and for a build request with [`--allow network.host` flag](https://docs.docker.com/engine/reference/commandline/buildx_build/#allow). { .warning } -## RUN --security +### RUN --security > **Note** > > Not yet available in stable syntax, use [`docker/dockerfile:1-labs`](#syntax) version. -### RUN --security=insecure +```dockerfile +RUN --security= +``` -With `--security=insecure`, builder runs the command without sandbox in insecure +The default security mode is `sandbox`. +With `--security=insecure`, the builder runs the command without sandbox in insecure mode, which allows to run flows requiring elevated privileges (e.g. containerd). This is equivalent to running `docker run --privileged`. @@ -878,6 +877,8 @@ This is equivalent to running `docker run --privileged`. > and for a build request with [`--allow security.insecure` flag](https://docs.docker.com/engine/reference/commandline/buildx_build/#allow). { .warning } +Default sandbox mode can be activated via `--security=sandbox`, but that is no-op. + #### Example: check entitlements ```dockerfile @@ -890,10 +891,6 @@ RUN --security=insecure cat /proc/self/status | grep CapEff #84 0.093 CapEff: 0000003fffffffff ``` -### RUN --security=sandbox - -Default sandbox mode can be activated via `--security=sandbox`, but that is no-op. - ## CMD The `CMD` instruction sets the command to be executed when running a container @@ -1135,33 +1132,22 @@ RUN apt-get update && apt-get install -y ... ## ADD -ADD has two forms: +ADD has two forms. +The latter form is required for paths containing whitespace. ```dockerfile -ADD [--chown=:] [--chmod= [--exclude=]... [--checksum=] ... -ADD [--chown=:] [--chmod= [--exclude=]... ["",... ""] +ADD [OPTIONS] ... +ADD [OPTIONS] ["", ... ""] ``` -The latter form is required for paths containing whitespace. +The available `[OPTIONS]` are: -> **Note** -> -> The `--chown` and `--chmod` features are only supported on Dockerfiles used to build Linux containers, -> and doesn't work on Windows containers. Since user and group ownership concepts do -> not translate between Linux and Windows, the use of `/etc/passwd` and `/etc/group` for -> translating user and group names to IDs restricts this feature to only be viable -> for Linux OS-based containers. - -> **Note** -> -> `--chmod` is supported since [Dockerfile 1.3](https://docs.docker.com/build/buildkit/dockerfile-frontend/). -> Only octal notation is currently supported. Non-octal support is tracked in -> [moby/buildkit#1951](https://github.com/moby/buildkit/issues/1951). - -> **Note** -> -> The `--exclude` option can be specified multiple times and cause files matching its patterns not to be copied, -> even if the files paths match the pattern specified in ``. This feature requires the build tag `dfexcludepatterns`. +- [`--keep-git-dir`](#add---keep-git-dir) +- [`--checksum`](#add---checksum) +- [`--chown`](#add---chown---chmod) +- [`--chmod`](#add---chown---chmod) +- [`--link`](#add---link) +- [`--exclude`](#add---exclude) The `ADD` instruction copies new files, directories or remote file URLs from `` and adds them to the filesystem of the image at the path ``. @@ -1170,16 +1156,16 @@ Multiple `` resources may be specified but if they are files or directories, their paths are interpreted as relative to the source of the context of the build. -Each `` and `` may contain wildcards and matching will be done using Go's +Each `` may contain wildcards and matching will be done using Go's [filepath.Match](https://golang.org/pkg/path/filepath#Match) rules. For example: -To add all files starting with "hom", excluding all files with `.txt` and `.md` extensions: +To add all files in the root of the build context starting with "hom": ```dockerfile -ADD --exclude=*.txt --exclude=*.md hom* /mydir/ +ADD hom* /mydir/ ``` -In the example below, `?` is replaced with any single character, e.g., "home.txt". +In the following example, `?` is a single-character wildcard, matching e.g. "home.txt". ```dockerfile ADD hom?.txt /mydir/ @@ -1209,30 +1195,6 @@ named `arr[0].txt`, use the following; ADD arr[[]0].txt /mydir/ ``` -All new files and directories are created with a UID and GID of 0, unless the -optional `--chown` flag specifies a given username, groupname, or UID/GID -combination to request specific ownership of the content added. The -format of the `--chown` flag allows for either username and groupname strings -or direct integer UID and GID in any combination. Providing a username without -groupname or a UID without GID will use the same numeric UID as the GID. If a -username or groupname is provided, the container's root filesystem -`/etc/passwd` and `/etc/group` files will be used to perform the translation -from name to integer UID or GID respectively. The following examples show -valid definitions for the `--chown` flag: - -```dockerfile -ADD --chown=55:mygroup files* /somedir/ -ADD --chown=bin files* /somedir/ -ADD --chown=1 files* /somedir/ -ADD --chown=10:11 files* /somedir/ -ADD --chown=myuser:mygroup --chmod=655 files* /somedir/ -``` - -If the container root filesystem doesn't contain either `/etc/passwd` or -`/etc/group` files and either user or group names are used in the `--chown` -flag, the build will fail on the `ADD` operation. Using numeric IDs requires -no lookup and doesn't depend on container root filesystem content. - In the case where `` is a remote file URL, the destination will have permissions of 600. If the remote file being retrieved has an HTTP `Last-Modified` header, the timestamp from that header will be used @@ -1316,79 +1278,87 @@ doesn't support authentication. - If `` doesn't exist, it's created, along with all missing directories in its path. -### Verifying a remote file checksum `ADD --checksum= ` +### Adding private Git repositories -The checksum of a remote file can be verified with the `--checksum` flag: +To add a private repository via SSH, create a Dockerfile with the following form: ```dockerfile -ADD --checksum=sha256:24454f830cdb571e2c4ad15481119c43b3cafd48dd869a9b2945d1036d1dc68d https://mirrors.edge.kernel.org/pub/linux/kernel/Historic/linux-0.01.tar.gz / +# syntax=docker/dockerfile:1 +FROM alpine +ADD git@git.example.com:foo/bar.git /bar ``` -The `--checksum` flag only supports HTTP sources currently. +This Dockerfile can be built with `docker build --ssh` or `buildctl build --ssh`, e.g., -### Adding a Git repository `ADD ` +```console +$ docker build --ssh default +``` + +```console +$ buildctl build --frontend=dockerfile.v0 --local context=. --local dockerfile=. --ssh default +``` -This form allows adding a Git repository to an image directly, without using the `git` command inside the image: +### ADD --keep-git-dir ```dockerfile -ADD [--keep-git-dir=] +ADD [--keep-git-dir=] ... ``` +When `` is the HTTP or SSH address of a remote Git repository, +BuildKit adds the contents of the Git repository to the image +excluding the `.git` directory by default. + +The `--keep-git-dir=true` flag lets you preserve the `.git` directory. + ```dockerfile # syntax=docker/dockerfile:1 FROM alpine ADD --keep-git-dir=true https://github.com/moby/buildkit.git#v0.10.1 /buildkit ``` -The `--keep-git-dir=true` flag adds the `.git` directory. This flag defaults to false. +### ADD --checksum -### Adding a private git repository +```dockerfile +ADD [--checksum=] ... +``` -To add a private repo via SSH, create a Dockerfile with the following form: +The `--checksum` flag lets you verify the checksum of a remote resource: ```dockerfile -# syntax=docker/dockerfile:1 -FROM alpine -ADD git@git.example.com:foo/bar.git /bar +ADD --checksum=sha256:24454f830cdb571e2c4ad15481119c43b3cafd48dd869a9b2945d1036d1dc68d https://mirrors.edge.kernel.org/pub/linux/kernel/Historic/linux-0.01.tar.gz / ``` -This Dockerfile can be built with `docker build --ssh` or `buildctl build --ssh`, e.g., +The `--checksum` flag only supports HTTP sources currently. -```console -$ docker build --ssh default -``` +### ADD --chown --chmod -```console -$ buildctl build --frontend=dockerfile.v0 --local context=. --local dockerfile=. --ssh default -``` +See [`COPY --chown --chmod`](#copy---chown---chmod). -## ADD --link +### ADD --link See [`COPY --link`](#copy---link). +### ADD --exclude + +See [`COPY --exclude`](#copy---exclude). + ## COPY -COPY has two forms: +COPY has two forms. +The latter form is required for paths containing whitespace. ```dockerfile -COPY [--chown=:] [--chmod=] [--exclude=]... ... -COPY [--chown=:] [--chmod=] [--exclude=]... ["",... ""] +COPY [OPTIONS] ... +COPY [OPTIONS] ["", ... ""] ``` -This latter form is required for paths containing whitespace +The available `[OPTIONS]` are: -> **Note** -> -> The `--chown` and `--chmod` features are only supported on Dockerfiles used to build Linux containers, -> and doesn't work on Windows containers. Since user and group ownership concepts do -> not translate between Linux and Windows, the use of `/etc/passwd` and `/etc/group` for -> translating user and group names to IDs restricts this feature to only be viable for -> Linux OS-based containers. - -> **Note** -> -> The `--exclude` option can be specified multiple times and cause files matching its patterns not to be copied, -> even if the files paths match the pattern specified in ``. This feature requires the build tag `dfexcludepatterns`. +- [`--chown`](#copy---chown---chmod) +- [`--chmod`](#copy---chown---chmod) +- [`--link`](#copy---link) +- [`--parents`](#copy---parents) +- [`--exclude`](#copy---exclude) The `COPY` instruction copies new files or directories from `` and adds them to the filesystem of the container at the path ``. @@ -1400,13 +1370,13 @@ of the build. Each `` may contain wildcards and matching will be done using Go's [filepath.Match](https://golang.org/pkg/path/filepath#Match) rules. For example: -To add all files starting with "hom", excluding all files with `.txt` and `.md` extensions: +To add all files in the root of the build context starting with "hom": ```dockerfile -COPY --exclude=*.txt --exclude=*.md hom* /mydir/ +COPY hom* /mydir/ ``` -In the example below, `?` is replaced with any single character, e.g., "home.txt". +In the following example, `?` is a single-character wildcard, matching e.g. "home.txt". ```dockerfile COPY hom?.txt /mydir/ @@ -1436,30 +1406,6 @@ named `arr[0].txt`, use the following; COPY arr[[]0].txt /mydir/ ``` -All new files and directories are created with a UID and GID of 0, unless the -optional `--chown` flag specifies a given username, groupname, or UID/GID -combination to request specific ownership of the copied content. The -format of the `--chown` flag allows for either username and groupname strings -or direct integer UID and GID in any combination. Providing a username without -groupname or a UID without GID will use the same numeric UID as the GID. If a -username or groupname is provided, the container's root filesystem -`/etc/passwd` and `/etc/group` files will be used to perform the translation -from name to integer UID or GID respectively. The following examples show -valid definitions for the `--chown` flag: - -```dockerfile -COPY --chown=55:mygroup files* /somedir/ -COPY --chown=bin files* /somedir/ -COPY --chown=1 files* /somedir/ -COPY --chown=10:11 files* /somedir/ -COPY --chown=myuser:mygroup --chmod=644 files* /somedir/ -``` - -If the container root filesystem doesn't contain either `/etc/passwd` or -`/etc/group` files and either user or group names are used in the `--chown` -flag, the build will fail on the `COPY` operation. Using numeric IDs requires -no lookup and does not depend on container root filesystem content. - > **Note** > > If you build using STDIN (`docker build - < somefile`), there is no @@ -1509,11 +1455,52 @@ attempted to be used instead. > guide – Leverage build cache](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#leverage-build-cache) > for more information. -## COPY --link +### COPY --chown --chmod > **Note** > -> Added in [`docker/dockerfile:1.4`](#syntax) +> Only octal notation is currently supported. Non-octal support is tracked in +> [moby/buildkit#1951](https://github.com/moby/buildkit/issues/1951). + +```dockerfile +COPY [--chown=:] [--chmod= ...] ... +``` + +The `--chown` and `--chmod` features are only supported on Dockerfiles used to build Linux containers, +and doesn't work on Windows containers. Since user and group ownership concepts do +not translate between Linux and Windows, the use of `/etc/passwd` and `/etc/group` for +translating user and group names to IDs restricts this feature to only be viable for +Linux OS-based containers. + +All new files and directories are created with a UID and GID of 0, unless the +optional `--chown` flag specifies a given username, groupname, or UID/GID +combination to request specific ownership of the copied content. The +format of the `--chown` flag allows for either username and groupname strings +or direct integer UID and GID in any combination. Providing a username without +groupname or a UID without GID will use the same numeric UID as the GID. If a +username or groupname is provided, the container's root filesystem +`/etc/passwd` and `/etc/group` files will be used to perform the translation +from name to integer UID or GID respectively. The following examples show +valid definitions for the `--chown` flag: + +```dockerfile +COPY --chown=55:mygroup files* /somedir/ +COPY --chown=bin files* /somedir/ +COPY --chown=1 files* /somedir/ +COPY --chown=10:11 files* /somedir/ +COPY --chown=myuser:mygroup --chmod=644 files* /somedir/ +``` + +If the container root filesystem doesn't contain either `/etc/passwd` or +`/etc/group` files and either user or group names are used in the `--chown` +flag, the build will fail on the `COPY` operation. Using numeric IDs requires +no lookup and does not depend on container root filesystem content. + +### COPY --link + +```dockerfile +COPY [--link[=]] ... +``` Enabling this flag in `COPY` or `ADD` commands allows you to copy files with enhanced semantics where your files remain independent on their own layer and @@ -1544,7 +1531,7 @@ COPY /foo /bar and merging all the layers of both images together. -### Benefits of using `--link` +#### Benefits of using `--link` Use `--link` to reuse already built layers in subsequent builds with `--cache-from` even if the previous layers have changed. This is especially @@ -1565,7 +1552,7 @@ the files in the base image. In that case BuildKit will only build the layers for the `COPY` commands and push them to the registry directly on top of the layers of the base image. -### Incompatibilities with `--link=false` +#### Incompatibilities with `--link=false` When using `--link` the `COPY/ADD` commands are not allowed to read any files from the previous state. This means that if in previous state the destination @@ -1578,15 +1565,15 @@ path, using `--link` is always recommended. The performance of `--link` is equivalent or better than the default behavior and, it creates much better conditions for cache reuse. -## COPY --parents +### COPY --parents > **Note** > > Available in [`docker/dockerfile-upstream:master-labs`](#syntax). -> Will be included in `docker/dockerfile:1.6-labs`. +> Will be included in `docker/dockerfile:1.7-labs`. ```dockerfile -COPY [--parents[=]] ... +COPY [--parents[=]] ... ``` The `--parents` flag preserves parent directories for `src` entries. This flag defaults to `false`. @@ -1617,6 +1604,37 @@ to keep the layer count in the resulting image as low as possible. Therefore, with the `--parents` flag, the Buildkit is capable of packing multiple `COPY` instructions together, keeping the directory structure intact. +### COPY --exclude + +> **Note** +> +> Available in [`docker/dockerfile-upstream:master-labs`](#syntax). +> Will be included in `docker/dockerfile:1.7-labs`. + +```dockerfile +COPY [--exclude= ...] ... +``` + +The `--exclude` flag lets you specify a path expression for files to be excluded. + +The path expression follows the same format as ``, +supporting wildcards and matching using Go's +[filepath.Match](https://golang.org/pkg/path/filepath#Match) rules. +For example, to add all files starting with "hom", excluding files with a `.txt` extension: + +```dockerfile +COPY --exclude=*.txt hom* /mydir/ +``` + +You can specify the `--exclude` option multiple times for a `COPY` instruction. +Multiple `--excludes` are files matching its patterns not to be copied, +even if the files paths match the pattern specified in ``. +To add all files starting with "hom", excluding files with either `.txt` or `.md` extensions: + +```dockerfile +COPY --exclude=*.txt --exclude=*.md hom* /mydir/ +``` + ## ENTRYPOINT An `ENTRYPOINT` allows you to configure a container that will run as an executable. @@ -2629,10 +2647,6 @@ required such as `zsh`, `csh`, `tcsh` and others. ## Here-Documents -> **Note** -> -> Added in [`docker/dockerfile:1.4`](#syntax) - Here-documents allow redirection of subsequent Dockerfile lines to the input of `RUN` or `COPY` commands. If such command contains a [here-document](https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_07_04) the Dockerfile considers the next lines until the line only containing a