Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

buildkit: Add support for --mount=type=tmpfs to allow mounting volatile memory instead of persistent storage. #3575

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 37 additions & 2 deletions docs/Containerfile.5.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,14 +94,49 @@ A Containerfile is similar to a Makefile.
# Executable form
RUN ["executable", "param1", "param2"]
```
**RUN mounts**

**RUN Secrets*
**--mount**=*type=TYPE,TYPE-SPECIFIC-OPTION[,...]*

Attach a filesystem mount to the container

Current supported mount TYPES are bind, and tmpfs.

e.g.

mount=type=bind,source=/path/on/host,destination=/path/in/container

mount=type=tmpfs,tmpfs-size=512M,destination=/path/in/container

Common Options:

· src, source: mount source spec for bind and volume. Mandatory for bind.

· dst, destination, target: mount destination spec.

· ro, read-only: true or false (default).

Options specific to bind:

· bind-propagation: shared, slave, private, rshared, rslave, or rprivate(default). See also mount(2).

. bind-nonrecursive: do not setup a recursive bind mount. By default it is recursive.

Options specific to tmpfs:

· tmpfs-size: Size of the tmpfs mount in bytes. Unlimited by default in Linux.

· tmpfs-mode: File mode of the tmpfs in octal. (e.g. 700 or 0700.) Defaults to 1777 in Linux.

· tmpcopyup: Path that is shadowed by the tmpfs mount is recursively copied up to the tmpfs itself.

**RUN Secrets**

The RUN command has a feature to allow the passing of secret information into the image build. These secrets files can be used during the RUN command but are not committed to the final image. The `RUN` command supports the `--mount` option to identify the secret file. A secret file from the host is mounted into the container while the image is being built.

Container engines pass secret the secret file into the build using the `--secret` flag.

**RUN --mount* options:
**--mount**=*type=secret,TYPE-SPECIFIC-OPTION[,...]*

- `id` is the identifier to for the secret passed into the `buildah bud --secret` or `podman build --secret`. This identifier is associated with the RUN --mount identifier to use in the Containerfile.

Expand Down
2 changes: 2 additions & 0 deletions docs/buildah-run.1.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,8 @@ Current supported mount TYPES are bind, and tmpfs. <sup>[[1]](#Footnote1)</sup>

· tmpfs-mode: File mode of the tmpfs in octal. (e.g. 700 or 0700.) Defaults to 1777 in Linux.

· tmpcopyup: Path that is shadowed by the tmpfs mount is recursively copied up to the tmpfs itself.

**--network**, **--net**=*mode*

Sets the configuration for the network namespace for the container.
Expand Down
3 changes: 3 additions & 0 deletions pkg/parse/parse.go
Original file line number Diff line number Diff line change
Expand Up @@ -431,6 +431,9 @@ func GetTmpfsMount(args []string) (specs.Mount, error) {
case "readonly":
// Alias for "ro"
newMount.Options = append(newMount.Options, "ro")
case "tmpcopyup":
//the path that is shadowed by the tmpfs mount is recursively copied up to the tmpfs itself.
newMount.Options = append(newMount.Options, kv[0])
case "tmpfs-mode":
if len(kv) == 1 {
return newMount, errors.Wrapf(optionArgError, kv[0])
Expand Down
21 changes: 21 additions & 0 deletions run_linux.go
Original file line number Diff line number Diff line change
Expand Up @@ -2400,6 +2400,13 @@ func (b *Builder) runSetupRunMounts(mounts []string, secrets map[string]string,
}
finalMounts = append(finalMounts, *mount)
mountTargets = append(mountTargets, mount.Destination)
case "tmpfs":
mount, err := b.getTmpfsMount(tokens, rootUID, rootGID, processUID, processGID)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious does this default to tmpcopyup?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rhatdan No buildah mount does not adds tmpcopyup during builds by default but we can add it an as option.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added tmpcopyup as a new option in separate commit.

if err != nil {
return nil, nil, err
}
finalMounts = append(finalMounts, *mount)
mountTargets = append(mountTargets, mount.Destination)
default:
return nil, nil, errors.Errorf("invalid mount type %q", kv[1])
}
Expand Down Expand Up @@ -2429,6 +2436,20 @@ func (b *Builder) getBindMount(tokens []string, contextDir string, rootUID, root
return &volumes[0], nil
}

func (b *Builder) getTmpfsMount(tokens []string, rootUID, rootGID, processUID, processGID int) (*spec.Mount, error) {
var optionMounts []specs.Mount
mount, err := parse.GetTmpfsMount(tokens)
if err != nil {
return nil, err
}
optionMounts = append(optionMounts, mount)
volumes, err := b.runSetupVolumeMounts(b.MountLabel, nil, optionMounts, rootUID, rootGID, processUID, processGID)
if err != nil {
return nil, err
}
return &volumes[0], nil
}

func getSecretMount(tokens []string, secrets map[string]string, mountlabel string, containerWorkingDir string, uidmap []spec.LinuxIDMapping, gidmap []spec.LinuxIDMapping) (*spec.Mount, error) {
errInvalidSyntax := errors.New("secret should have syntax id=id[,target=path,required=bool,mode=uint,uid=uint,gid=uint")
if len(tokens) == 0 {
Expand Down
17 changes: 17 additions & 0 deletions tests/bud.bats
Original file line number Diff line number Diff line change
Expand Up @@ -3550,3 +3550,20 @@ _EOF
expect_output --substring "world"
run_buildah rmi -f testbud
}

@test "bud-with-mount-with-tmpfs-like-buildkit" {
skip_if_no_runtime
skip_if_in_container
# tmpfs mount: target should be available on container without creating any special directory on container
run_buildah build -t testbud --signature-policy ${TESTSDIR}/policy.json -f ${TESTSDIR}/bud/buildkit-mount/Dockerfiletmpfs
[ "$status" -eq 0 ]
run_buildah rmi -f testbud
}

@test "bud-with-mount-with-tmpfs-with-copyup-like-buildkit" {
skip_if_no_runtime
skip_if_in_container
run_buildah build -t testbud --signature-policy ${TESTSDIR}/policy.json -f ${TESTSDIR}/bud/buildkit-mount/Dockerfiletmpfscopyup
expect_output --substring "certs"
run_buildah rmi -f testbud
}
4 changes: 4 additions & 0 deletions tests/bud/buildkit-mount/Dockerfiletmpfs
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
FROM alpine

# As a baseline, this should succeed without creating any directory on container
RUN --mount=type=tmpfs,target=/var/tmpfs-not-empty touch /var/tmpfs-not-empty/hello
4 changes: 4 additions & 0 deletions tests/bud/buildkit-mount/Dockerfiletmpfscopyup
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
FROM alpine

# As a baseline, this should succeed without creating any directory on container
RUN --mount=type=tmpfs,target=/etc/ssl,tmpcopyup ls /etc/ssl