Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

buildctl: new CLI ("Option C+") #807

Merged
merged 1 commit into from
Mar 8, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 15 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ During development, Dockerfile frontend (dockerfile.v0) is also part of the Buil

```
buildctl build --frontend=dockerfile.v0 --local context=. --local dockerfile=.
buildctl build --frontend=dockerfile.v0 --local context=. --local dockerfile=. --frontend-opt target=foo --frontend-opt build-arg:foo=bar
buildctl build --frontend=dockerfile.v0 --local context=. --local dockerfile=. --opt target=foo --opt build-arg:foo=bar
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bit on the fence on the --frontend-opt -> --opt change, as it's not clear to me that they are related to each other. 😅

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why should you as a user care if they are related or not?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To understand that an option is related to the frontend, not to (e.g.) the --output.

If we drop the prefix, should we also drop the --export- prefix from --export-cache?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't say they are strictly related to the frontend. They are more like options for the build request, while the other options are for a specific component (exporter, cache) or are cli side (local, secret).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah; gotcha - guess I got confused because they previously were prefixed

```

`--local` exposes local source files from client to the builder. `context` and `dockerfile` are the names Dockerfile frontend looks for build context and Dockerfile location.
Expand All @@ -146,31 +146,31 @@ docker inspect myimage
External versions of the Dockerfile frontend are pushed to https://hub.docker.com/r/docker/dockerfile-upstream and https://hub.docker.com/r/docker/dockerfile and can be used with the gateway frontend. The source for the external frontend is currently located in `./frontend/dockerfile/cmd/dockerfile-frontend` but will move out of this repository in the future ([#163](https://github.com/moby/buildkit/issues/163)). For automatic build from master branch of this repository `docker/dockerfile-upsteam:master` or `docker/dockerfile-upstream:master-experimental` image can be used.

```
buildctl build --frontend=gateway.v0 --frontend-opt=source=docker/dockerfile --local context=. --local dockerfile=.
buildctl build --frontend gateway.v0 --frontend-opt=source=docker/dockerfile --frontend-opt=context=git://github.com/moby/moby --frontend-opt build-arg:APT_MIRROR=cdn-fastly.deb.debian.org
buildctl build --frontend gateway.v0 --opt source=docker/dockerfile --local context=. --local dockerfile=.
buildctl build --frontend gateway.v0 --opt source=docker/dockerfile --opt context=git://github.com/moby/moby --opt build-arg:APT_MIRROR=cdn-fastly.deb.debian.org
````

##### Building a Dockerfile with experimental features like `RUN --mount=type=(bind|cache|tmpfs|secret|ssh)`

See [`frontend/dockerfile/docs/experimental.md`](frontend/dockerfile/docs/experimental.md).

### Exporters
### Output

By default, the build result and intermediate cache will only remain internally in BuildKit. Exporter needs to be specified to retrieve the result.
By default, the build result and intermediate cache will only remain internally in BuildKit. An output needs to be specified to retrieve the result.

##### Exporting resulting image to containerd

The containerd worker needs to be used

```
buildctl build ... --exporter=image --exporter-opt name=docker.io/username/image
buildctl build ... --output type=image,name=docker.io/username/image
ctr --namespace=buildkit images ls
```

##### Push resulting image to registry

```
buildctl build ... --exporter=image --exporter-opt name=docker.io/username/image --exporter-opt push=true
buildctl build ... --output type=image,name=docker.io/username/image,push=true
```

If credentials are required, `buildctl` will attempt to read Docker configuration file.
Expand All @@ -181,21 +181,21 @@ If credentials are required, `buildctl` will attempt to read Docker configuratio
The local client will copy the files directly to the client. This is useful if BuildKit is being used for building something else than container images.

```
buildctl build ... --exporter=local --exporter-opt output=path/to/output-dir
buildctl build ... --output type=local,dest=path/to/output-dir
```

##### Exporting built image to Docker

```
# exported tarball is also compatible with OCI spec
buildctl build ... --exporter=docker --exporter-opt name=myimage | docker load
buildctl build ... --output type=docker,name=myimage | docker load
```

##### Exporting [OCI Image Format](https://github.com/opencontainers/image-spec) tarball to client

```
buildctl build ... --exporter=oci --exporter-opt output=path/to/output.tar
buildctl build ... --exporter=oci > output.tar
buildctl build ... --output type=oci,dest=path/to/output.tar
buildctl build ... --output type=oci > output.tar
```

### Exporting/Importing build cache (not image itself)
Expand All @@ -210,8 +210,8 @@ buildctl build ... --import-cache type=registry,ref=localhost:5000/myrepo:buildc
#### To/From local filesystem

```
buildctl build ... --export-cache type=local,store=path/to/input-dir
buildctl build ... --import-cache type=local,store=path/to/output-dir
buildctl build ... --export-cache type=local,src=path/to/input-dir
buildctl build ... --import-cache type=local,dest=path/to/output-dir
```

The directory layout conforms to OCI Image Spec v1.0.
Expand All @@ -220,11 +220,11 @@ The directory layout conforms to OCI Image Spec v1.0.
* `mode=min` (default): only export layers for the resulting image
* `mode=max`: export all the layers of all intermediate steps
* `ref=docker.io/user/image:tag`: reference for `registry` cache exporter
* `store=path/to/output-dir`: directory for `local` cache exporter
* `src=path/to/output-dir`: directory for `local` cache exporter

#### `--import-cache` options
* `ref=docker.io/user/image:tag`: reference for `registry` cache importer
* `store=path/to/input-dir`: directory for `local` cache importer
* `dest=path/to/input-dir`: directory for `local` cache importer
* `digest=sha256:deadbeef`: digest of the manifest list to import for `local` cache importer. Defaults to the digest of "latest" tag in `index.json`

### Other
Expand Down
11 changes: 6 additions & 5 deletions cache/remotecache/local/local.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,16 +15,17 @@ import (

const (
attrDigest = "digest"
attrStore = "store"
attrSrc = "src"
attrDest = "dest"
contentStoreIDPrefix = "local:"
)

// ResolveCacheExporterFunc for "local" cache exporter.
func ResolveCacheExporterFunc(sm *session.Manager) remotecache.ResolveCacheExporterFunc {
return func(ctx context.Context, attrs map[string]string) (remotecache.Exporter, error) {
store := attrs[attrStore]
store := attrs[attrDest]
if store == "" {
return nil, errors.New("local cache exporter requires store")
return nil, errors.New("local cache exporter requires dest")
}
csID := contentStoreIDPrefix + store
cs, err := getContentStore(ctx, sm, csID)
Expand All @@ -43,9 +44,9 @@ func ResolveCacheImporterFunc(sm *session.Manager) remotecache.ResolveCacheImpor
return nil, specs.Descriptor{}, errors.New("local cache importer requires explicit digest")
}
dgst := digest.Digest(dgstStr)
store := attrs[attrStore]
store := attrs[attrSrc]
if store == "" {
return nil, specs.Descriptor{}, errors.New("local cache importer requires store")
return nil, specs.Descriptor{}, errors.New("local cache importer requires src")
}
csID := contentStoreIDPrefix + store
cs, err := getContentStore(ctx, sm, csID)
Expand Down
8 changes: 6 additions & 2 deletions client/build_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -85,8 +85,12 @@ func testClientGatewaySolve(t *testing.T, sb integration.Sandbox) {
testStr := "This is a test"

_, err = c.Build(ctx, SolveOpt{
Exporter: ExporterLocal,
ExporterOutputDir: tmpdir,
Exports: []ExportEntry{
{
Type: ExporterLocal,
OutputDir: tmpdir,
},
},
FrontendAttrs: map[string]string{
optKey: testStr,
},
Expand Down
Loading