From d7e2c4b2031f8fff307562e89de8db01cc8fbdbb Mon Sep 17 00:00:00 2001 From: Halvard Skogsrud Date: Mon, 29 Mar 2021 23:21:35 +1100 Subject: [PATCH 1/7] Add ko builder design proposal first draft --- docs/design_proposals/ko-builder.md | 215 ++++++++++++++++++++++++++++ 1 file changed, 215 insertions(+) create mode 100644 docs/design_proposals/ko-builder.md diff --git a/docs/design_proposals/ko-builder.md b/docs/design_proposals/ko-builder.md new file mode 100644 index 00000000000..f62eaa92e04 --- /dev/null +++ b/docs/design_proposals/ko-builder.md @@ -0,0 +1,215 @@ +# ko builder + +* Author(s): Halvard Skogsrud (@halvards) +* Design Shepherd: \ +* Date: 2021-03-29 +* Status: Draft + +## Objectives + +Fast, standardized, reproducible, configuration-less, Docker-less, and +secure-by-default container image builds for Go apps. + +## Background + +[ko](https://github.com/google/ko) is a container image builder for Go. It's +[fast](https://cloud.google.com/blog/topics/developers-practitioners/ship-your-go-applications-faster-cloud-run-ko), +doesn't use a `Dockerfile` or rely on the Docker daemon, and uses +[distroless](https://github.com/GoogleContainerTools/distroless) base images by +default. ko is to Go apps what +[Jib](https://github.com/GoogleContainerTools/jib) is to JVM-based apps +(approximately). + +The [Knative](https://knative.dev/) and [Tekton](https://tekton.dev/) open +source projects use ko. + +## Proposal + +This proposal adds a new `ko` builder to Skaffold, based on the `ko publish` +command. The integration does _not_ include other ko functionality related to +[rendering](https://github.com/google/ko#ko-resolve) manifests, +[deploying](https://github.com/google/ko#ko-apply) to Kubernetes clusters, and +[file watching](https://github.com/google/ko/blob/f7df8106196518df5c6c35432843421e33990329/pkg/commands/resolver.go#L240). + +Compared to ... + +- [the Cloud Native buildpacks builder](https://skaffold.dev/docs/pipeline-stages/builders/buildpacks/), + the ko builder is + [fast](https://cloud.google.com/blog/topics/developers-practitioners/ship-your-go-applications-faster-cloud-run-ko), + doesn't require Docker, and uses a default base image that has a small attack + surface ([distroless](https://github.com/GoogleContainerTools/distroless)). + +- [the Docker builder](https://skaffold.dev/docs/pipeline-stages/builders/docker/), + the ko builder standardizes builds, avoiding artisanal + [snowflake](https://martinfowler.com/bliki/SnowflakeServer.html) + `Dockerfile`s. It also doesn't require the Docker daemon, so builds can + run in security-constrained environments. + +- [the kaniko builder](https://skaffold.dev/docs/pipeline-stages/builders/docker/#dockerfile-in-cluster-with-kaniko), + the ko builder doesn't need a Kubernetes cluster, and avoids the + previously-mentioned artisanal `Dockerfile`s. + +- [the Bazel builder](https://skaffold.dev/docs/pipeline-stages/builders/bazel/), + the ko builder doesn't require users to adopt Bazel. However, users who + already use Bazel for their Go app should use the Bazel builder. + +- [the custom builder](https://skaffold.dev/docs/pipeline-stages/builders/custom/), + the ko builder is portable: + + 1. The Skaffold config can be shared with other developers and ops teams, + and used in CI/CD pipelines, without requiring installing additional + tools such as Docker or + [crane](https://github.com/google/go-containerregistry/blob/main/cmd/crane/README.md) + (or even ko, depending on how the builder is implemented). This eases the + path to adoption of Skaffold and reduces friction for users, both for + local development, and for anyone using Skaffold in CI/CD pipelines. + + 2. The ko builder doesn't require running custom shell scripts. This means + more standardized builds, a desirable trait for enterprise users. + +The ko builder supports and enhances these Skaffold +[features](https://skaffold.dev/docs/): + +- _fast local workflow_: building with ko is + [fast](https://cloud.google.com/blog/topics/developers-practitioners/ship-your-go-applications-faster-cloud-run-ko). + +- _share with other developers_: no additional tools are required to + `skaffold run` with the ko builder, not even Docker. + +- works great as a _CI/CD building block_: when using the ko builder, pipeline + steps can run using the default Skaffold container image, without + installing additional tools or keeping toolchain versions in sync across + local development and CI/CD. + +## Design + +Adding the ko builder requires making config changes to the Skaffold schema. + +1. Add an entry to BuilderType enum in `proto/enums/enums.proto`: + + ```proto + // ko Builder + KO = 7; + ``` + +2. Add a `KoArtifact` type: + + ```go + type KoArtifact struct { + // Flags are additional build flags passed to the builder. + // For example: `["--platform=linux/amd64,linux/arm64"]`. + Flags []string `yaml:"args,omitempty"` + + // BaseImage overrides the default ko base image. + // Corresponds to, and overrides, the `defaultBaseImage` in `.ko.yaml`. + BaseImage string `yaml:"fromImage,omitempty"` + } + ``` + +3. Add a `KoArtifact` field to the `ArtifactType` struct: + + ```go + type ArtifactType struct { + // KoArtifact builds images using [ko](https://github.com/google/ko). + KoArtifact *KoArtifact `yaml:"ko,omitempty" yamltags:"oneOf=artifact"` + } + ``` + +Example basic config, this will be sufficient for many users: + +```yaml +apiVersion: skaffold/v2beta14 +kind: Config +build: + artifacts: + - image: ko://example.com/helloworld + ko: {} +``` + +The value of the `image` field is the Go import path of the app entry point, +[prefixed by `ko://`](https://github.com/google/ko/pull/58). + +ko requires setting a +[`KO_DOCKER_REPO`](https://github.com/google/ko#choose-destination) +environment variable. This determines where container images are pushed. +The Skaffold +[default repo](https://skaffold.dev/docs/environment/image-registries/) +maps directly to this value. + +### Open Questions + +1. Should Skaffold embed ko (as a Go module), or shell out? + + Benefits of embedding: + + - Skaffold can pin the ko version it supports in its `go.mod` file. Users + wouldn't raise bugs/issues for incompatible version pairings of Skaffold + and ko. + + - Reduce toolchain maintenance toil for users. Skaffold users wouldn't need + to synchronize ko versions used by different team members or in their CI + build, since the Skaffold version determines the ko version. + + - Portability. Skaffold+ko users only need one tool for their container + image building needs: the `skaffold` binary. (Plus the Go distribution, + of course.) The current `gcr.io/k8s-skaffold/skaffold` container image + could serve as a build and deploy image for CI/CD pipeline steps. + + Embedding ko would require some stability guarantees for the most important + interfaces that Skaffold would use: + [`build.Interface`](https://github.com/google/ko/blob/82cabb40bae577ce3bc016e5939fd85889538e8b/pkg/build/build.go#L24) + and + [`publish.Interface`](https://github.com/google/ko/blob/82cabb40bae577ce3bc016e5939fd85889538e8b/pkg/publish/publish.go#L24), + or others? + + Benefits of shelling out: + + - It's an established pattern used by other Skaffold builders. + + - It would allow Skaffold to support a range of ko versions. On the other + hand, these versions would need to be tracked and documented. + + - No need to resolve dependency version differences between Skaffold and + ko. + + - If a new ko version provided a significant bug fix, there would be no + need to release a new version of Skaffold for this fix. + + Shelling out to ko would require some stability guarantees for the + `ko publish` subcommand. + + Suggest embedding as a Go module. __Not Yet Resolved__ + +2. Should the ko builder be the default for `skaffold init`, instead of + buildpacks, for Go apps, when there's no Dockerfile and no Bazel workspace + file? + + Suggest yes, to make Skaffold a compelling choice for Go developers. + __Not Yet Resolved__ + +3. Should Skaffold use base image settings from + [`.ko.yaml`](https://github.com/google/ko#configuration) if the ko builder + definition in `skaffold.yaml` doesn't specify a base image? + + Suggest yes to simplify adoption of Skaffold for existing ko users. + __Not Yet Resolved__ + +## -- Sections below haven't been fleshed out -- + +## Implementation plan + +1. TBC + +## Integration test plan + +Please describe what new test cases you are going to consider. + +1. Unit and integration tests for ko builder, similar to other builders. + +2. Test that the ko flag + [`--disable-optimization`](https://github.com/google/ko/blob/f7df8106196518df5c6c35432843421e33990329/pkg/commands/options/build.go#L34) + is added for debugging. + +3. File sync testing? + +4. Add ko example to the `examples` directory. From ca60ac68b0718f11d5639b91d1105d5e7dd48b68 Mon Sep 17 00:00:00 2001 From: Halvard Skogsrud Date: Tue, 30 Mar 2021 19:06:15 +1100 Subject: [PATCH 2/7] Update ko builder proposal based on feedback Updates based on feedback from @briandealwis --- docs/design_proposals/ko-builder.md | 43 +++++++++++++++++++++++------ 1 file changed, 34 insertions(+), 9 deletions(-) diff --git a/docs/design_proposals/ko-builder.md b/docs/design_proposals/ko-builder.md index f62eaa92e04..9b4adaf785e 100644 --- a/docs/design_proposals/ko-builder.md +++ b/docs/design_proposals/ko-builder.md @@ -45,7 +45,7 @@ Compared to ... `Dockerfile`s. It also doesn't require the Docker daemon, so builds can run in security-constrained environments. -- [the kaniko builder](https://skaffold.dev/docs/pipeline-stages/builders/docker/#dockerfile-in-cluster-with-kaniko), +- [the Kaniko builder](https://skaffold.dev/docs/pipeline-stages/builders/docker/#dockerfile-in-cluster-with-kaniko), the ko builder doesn't need a Kubernetes cluster, and avoids the previously-mentioned artisanal `Dockerfile`s. @@ -57,8 +57,8 @@ Compared to ... the ko builder is portable: 1. The Skaffold config can be shared with other developers and ops teams, - and used in CI/CD pipelines, without requiring installing additional - tools such as Docker or + and used in CI/CD pipelines, without requiring users to install + additional tools such as Docker Engine or [crane](https://github.com/google/go-containerregistry/blob/main/cmd/crane/README.md) (or even ko, depending on how the builder is implemented). This eases the path to adoption of Skaffold and reduces friction for users, both for @@ -74,7 +74,9 @@ The ko builder supports and enhances these Skaffold [fast](https://cloud.google.com/blog/topics/developers-practitioners/ship-your-go-applications-faster-cloud-run-ko). - _share with other developers_: no additional tools are required to - `skaffold run` with the ko builder, not even Docker. + `skaffold run` with the ko builder, not even Docker. Though if we don't embed + ko in Skaffold, ko will be a tool that all developers in a team would have to + install. - works great as a _CI/CD building block_: when using the ko builder, pipeline steps can run using the default Skaffold container image, without @@ -96,13 +98,30 @@ Adding the ko builder requires making config changes to the Skaffold schema. ```go type KoArtifact struct { - // Flags are additional build flags passed to the builder. - // For example: `["--platform=linux/amd64,linux/arm64"]`. - Flags []string `yaml:"args,omitempty"` + // Annotations are key-value string pairs to add to the image manifest. + // Also known as `LABEL` in `Dockerfile`s. + // Ref: https://github.com/opencontainers/image-spec/blob/master/annotations.md + Annotations map[string]string `yaml:"annotations,omitempty"` // BaseImage overrides the default ko base image. // Corresponds to, and overrides, the `defaultBaseImage` in `.ko.yaml`. BaseImage string `yaml:"fromImage,omitempty"` + + // Env are environment variables, in the `key=value` form, passed to the build. + // For example: `CGO_ENABLED=1`. + Env []string `yaml:"env,omitempty"` + + // Platforms is the list of platforms to build images for. Each platform + // is of the format `os/arch[/variant]`, e.g., `linux/amd64`. + // By default, the ko builder builds for `all` platforms supported by the + // base image. + Platforms []string `yaml:"platforms,omitempty"` + + // SourceDateEpoch is the `created` time of the container image. + // Specify as the number of seconds since January 1st 1970, 00:00 UTC. + // You can override this value by setting the `SOURCE_DATE_EPOCH` + // environment variable. + SourceDateEpoch unit64 `yaml:"sourceDateEpoch,omitempty"` } ``` @@ -155,8 +174,8 @@ maps directly to this value. of course.) The current `gcr.io/k8s-skaffold/skaffold` container image could serve as a build and deploy image for CI/CD pipeline steps. - Embedding ko would require some stability guarantees for the most important - interfaces that Skaffold would use: + Embedding ko would require some level of documented behavioural stability + guarantees for the most ko interfaces that Skaffold would use, such as [`build.Interface`](https://github.com/google/ko/blob/82cabb40bae577ce3bc016e5939fd85889538e8b/pkg/build/build.go#L24) and [`publish.Interface`](https://github.com/google/ko/blob/82cabb40bae577ce3bc016e5939fd85889538e8b/pkg/publish/publish.go#L24), @@ -194,6 +213,12 @@ maps directly to this value. Suggest yes to simplify adoption of Skaffold for existing ko users. __Not Yet Resolved__ +4. If a config value is set both as an environment variable, and as a config + value, which takes precedence? E.g., `ko.sourceDateEpoch` vs + `SOURCE_DATE_EPOCH`. + + Follow existing Skaffold pattern - is there one? __Not Yet Resolved__ + ## -- Sections below haven't been fleshed out -- ## Implementation plan From 7c138b67666fdf37959776b13a8ea0ec95c72bd5 Mon Sep 17 00:00:00 2001 From: Halvard Skogsrud Date: Fri, 9 Apr 2021 19:26:07 +1000 Subject: [PATCH 3/7] Add more comprehensive ko builder config example --- docs/design_proposals/ko-builder.md | 69 +++++++++++++++++++++++++---- 1 file changed, 61 insertions(+), 8 deletions(-) diff --git a/docs/design_proposals/ko-builder.md b/docs/design_proposals/ko-builder.md index 9b4adaf785e..dc0f97c89c0 100644 --- a/docs/design_proposals/ko-builder.md +++ b/docs/design_proposals/ko-builder.md @@ -97,22 +97,41 @@ Adding the ko builder requires making config changes to the Skaffold schema. 2. Add a `KoArtifact` type: ```go + // KoArtifact builds images using [ko](https://github.com/google/ko). type KoArtifact struct { - // Annotations are key-value string pairs to add to the image manifest. - // Also known as `LABEL` in `Dockerfile`s. - // Ref: https://github.com/opencontainers/image-spec/blob/master/annotations.md - Annotations map[string]string `yaml:"annotations,omitempty"` + // Asmflags are assembler flags passed to the builder. + Asmflags []string `yaml:"asmflags,omitempty"` // BaseImage overrides the default ko base image. // Corresponds to, and overrides, the `defaultBaseImage` in `.ko.yaml`. BaseImage string `yaml:"fromImage,omitempty"` + // Dependencies are the file dependencies that skaffold should watch for both rebuilding and file syncing for this artifact. + Dependencies *KoDependencies `yaml:"dependencies,omitempty"` + // Env are environment variables, in the `key=value` form, passed to the build. - // For example: `CGO_ENABLED=1`. + // These environment variables are only used at build time. + // They are _not_ set in the resulting container image. Env []string `yaml:"env,omitempty"` + // Flags are additional build flags passed to the builder. + // For example: `["-trimpath", "-v"]`. + Flags []string `yaml:"args,omitempty"` + + // Gcflags are Go compiler flags passed to the builder. + // For example: `["-m"]`. + Gcflags []string `yaml:"gcflags,omitempty"` + + // Labels are key-value string pairs to add to the image config. + // For example: `{"foo":"bar"}`. + Labels map[string]string `yaml:"labels,omitempty"` + + // Ldflags are linker flags passed to the builder. + // For example: `["-buildid=", "-s", "-w"]`. + Ldflags []string `yaml:"ldflags,omitempty"` + // Platforms is the list of platforms to build images for. Each platform - // is of the format `os/arch[/variant]`, e.g., `linux/amd64`. + // is of the format `os[/arch[/variant]]`, e.g., `linux/amd64`. // By default, the ko builder builds for `all` platforms supported by the // base image. Platforms []string `yaml:"platforms,omitempty"` @@ -121,7 +140,7 @@ Adding the ko builder requires making config changes to the Skaffold schema. // Specify as the number of seconds since January 1st 1970, 00:00 UTC. // You can override this value by setting the `SOURCE_DATE_EPOCH` // environment variable. - SourceDateEpoch unit64 `yaml:"sourceDateEpoch,omitempty"` + SourceDateEpoch uint64 `yaml:"sourceDateEpoch,omitempty"` } ``` @@ -141,13 +160,47 @@ apiVersion: skaffold/v2beta14 kind: Config build: artifacts: - - image: ko://example.com/helloworld + - image: ko://github.com/GoogleContainerTools/skaffold/examples/ko ko: {} ``` The value of the `image` field is the Go import path of the app entry point, [prefixed by `ko://`](https://github.com/google/ko/pull/58). +A more comprehensive example config: + +```yaml +apiVersion: skaffold/v2beta14 +kind: Config +build: + artifacts: + - image: ko://github.com/GoogleContainerTools/skaffold/examples/ko-complete + ko: + asmflags: [] + fromImage: gcr.io/distroless/static-debian10:nonroot + dependencies: + paths: + - go.mod + - "**.go" + env: [] + args: + - -trimpath + - -v + gcflags: + - -m + labels: + foo: bar + baz: frob + ldflags: + - -buildid= + - -s + - -w + platforms: + - linux/amd64 + - linux/arm64 + sourceDateEpoch: 946684800 +``` + ko requires setting a [`KO_DOCKER_REPO`](https://github.com/google/ko#choose-destination) environment variable. This determines where container images are pushed. From ade91ee5a9dd55b6022fcfdad6935a34181e6239 Mon Sep 17 00:00:00 2001 From: Halvard Skogsrud Date: Tue, 13 Apr 2021 18:11:52 +1000 Subject: [PATCH 4/7] Add implementation plan details --- docs/design_proposals/ko-builder.md | 78 +++++++++++++++++++++++++---- 1 file changed, 69 insertions(+), 9 deletions(-) diff --git a/docs/design_proposals/ko-builder.md b/docs/design_proposals/ko-builder.md index dc0f97c89c0..09f4b9f64f1 100644 --- a/docs/design_proposals/ko-builder.md +++ b/docs/design_proposals/ko-builder.md @@ -203,12 +203,11 @@ build: ko requires setting a [`KO_DOCKER_REPO`](https://github.com/google/ko#choose-destination) -environment variable. This determines where container images are pushed. -The Skaffold +environment variable to specify where container images are pushed. The Skaffold [default repo](https://skaffold.dev/docs/environment/image-registries/) maps directly to this value. -### Open Questions +### Open questions 1. Should Skaffold embed ko (as a Go module), or shell out? @@ -272,11 +271,71 @@ maps directly to this value. Follow existing Skaffold pattern - is there one? __Not Yet Resolved__ -## -- Sections below haven't been fleshed out -- - ## Implementation plan -1. TBC +1. Define interfaces in the ko codebase that allows ko to be used from + Skaffold without duplicating existing ko CLI code. + + Draft interface: + + ```go + type KoPublish interface { + func BuildAndPublish(context.Context, options.BuildOptions, options.PublishOptions, importpath string, imageNameWithTag string) + } + ``` + +2. Add ko builder with support for existing ko config options. Provide + this as an Alpha feature in an upcoming Skaffold release. + + Config options supported, all are optional: + + - `dependencies`, for Skaffold file watching. + - `env`, to support ko CLI users who currently set environment variables + such as `GOFLAGS` when running ko. + - `fromImage`, to override the default distroless base image + - `labels` + - `platforms` + - `sourceDateEpoch` + + Example `skaffold.yaml` supported at this stage: + + ```yaml + apiVersion: skaffold/v2beta14 + kind: Config + build: + artifacts: + - image: ko://github.com/GoogleContainerTools/skaffold/examples/ko + ko: + fromImage: gcr.io/distroless/static-debian10:nonroot + dependencies: + paths: + - go.mod + - "**.go" + env: [] + labels: + foo: bar + baz: frob + platforms: + - linux/amd64 + - linux/arm64 + sourceDateEpoch: 946684800 + ``` + +3. Implement support for additional config options in ko: + + - `args`, e.g., `-v`, `-trimpath` + - `asmflags` + - `gcflags` + - `ldflags` + + See related discussion in + [google/ko#316](https://github.com/google/ko/issues/316). + +4. Add support for the additional ko config options from step 2 in Skaffold. + Provide this as a feature in an upcoming Skaffold release. + + This will enable support for all the config options shown in the + comprehensive example above. ## Integration test plan @@ -284,10 +343,11 @@ Please describe what new test cases you are going to consider. 1. Unit and integration tests for ko builder, similar to other builders. + The integration tests should be written to catch situations such as where + changes to ko interfaces break the Skaffold ko builder. + 2. Test that the ko flag [`--disable-optimization`](https://github.com/google/ko/blob/f7df8106196518df5c6c35432843421e33990329/pkg/commands/options/build.go#L34) is added for debugging. -3. File sync testing? - -4. Add ko example to the `examples` directory. +3. Add ko example to the `examples` directory. From 958b971c61bfc22b2c6c3fd6cc850beaa052a3bd Mon Sep 17 00:00:00 2001 From: Halvard Skogsrud Date: Tue, 20 Apr 2021 16:59:24 +1000 Subject: [PATCH 5/7] Add initializer rank to ko builder proposal --- docs/design_proposals/ko-builder.md | 32 +++++++++++++++++++++++++++-- 1 file changed, 30 insertions(+), 2 deletions(-) diff --git a/docs/design_proposals/ko-builder.md b/docs/design_proposals/ko-builder.md index 09f4b9f64f1..7e0c8c709a8 100644 --- a/docs/design_proposals/ko-builder.md +++ b/docs/design_proposals/ko-builder.md @@ -153,6 +153,29 @@ Adding the ko builder requires making config changes to the Skaffold schema. } ``` +4. Define ko's position in the initializer builder rank in + `pkg/skaffold/initializer/build/resolve.go`. The proposal is to add it + ahead of the buildpacks builder: + + ```go + func builderRank(builder InitBuilder) int { + a := builder.ArtifactType("") + switch { + case a.DockerArtifact != nil: + return 1 + case a.JibArtifact != nil: + return 2 + case a.BazelArtifact != nil: + return 3 + case a.KoArtifact != nil: + return 4 + case a.BuildpackArtifact != nil: + return 5 + } + return 6 + } + ``` + Example basic config, this will be sufficient for many users: ```yaml @@ -198,7 +221,6 @@ build: platforms: - linux/amd64 - linux/arm64 - sourceDateEpoch: 946684800 ``` ko requires setting a @@ -265,7 +287,13 @@ maps directly to this value. Suggest yes to simplify adoption of Skaffold for existing ko users. __Not Yet Resolved__ -4. If a config value is set both as an environment variable, and as a config +4. Should the ko builder have a config option for + [`SOURCE_DATE_EPOCH`](https://reproducible-builds.org/specs/source-date-epoch/), + or should users specify the value via an environment variable? + + __Not Yet Resolved__ + +5. If a config value is set both as an environment variable, and as a config value, which takes precedence? E.g., `ko.sourceDateEpoch` vs `SOURCE_DATE_EPOCH`. From 1eed482530fbef51961c1221475b46f3a4e06646 Mon Sep 17 00:00:00 2001 From: Halvard Skogsrud Date: Wed, 21 Apr 2021 13:46:24 +1000 Subject: [PATCH 6/7] Ko builder proposal: add open questions --- docs/design_proposals/ko-builder.md | 46 +++++++++++++++++++++++++++-- 1 file changed, 43 insertions(+), 3 deletions(-) diff --git a/docs/design_proposals/ko-builder.md b/docs/design_proposals/ko-builder.md index 7e0c8c709a8..435a7de89fb 100644 --- a/docs/design_proposals/ko-builder.md +++ b/docs/design_proposals/ko-builder.md @@ -155,7 +155,7 @@ Adding the ko builder requires making config changes to the Skaffold schema. 4. Define ko's position in the initializer builder rank in `pkg/skaffold/initializer/build/resolve.go`. The proposal is to add it - ahead of the buildpacks builder: + at the end, after the buildpacks builder: ```go func builderRank(builder InitBuilder) int { @@ -167,15 +167,39 @@ Adding the ko builder requires making config changes to the Skaffold schema. return 2 case a.BazelArtifact != nil: return 3 - case a.KoArtifact != nil: - return 4 case a.BuildpackArtifact != nil: + return 4 + case a.KoArtifact != nil: return 5 } + return 6 } ``` +5. Add `KO` to the `BuilderType` enum in `proto/enums/enums.proto`: + + ```proto + enum BuilderType { + // Could not determine builder type + UNKNOWN_BUILDER_TYPE = 0; + // JIB Builder + JIB = 1; + // Bazel Builder + BAZEL = 2; + // Buildpacks Builder + BUILDPACKS = 3; + // Custom Builder + CUSTOM = 4; + // Kaniko Builder + KANIKO = 5; + // Docker Builder + DOCKER = 6; + // Ko Builder + KO = 7; + } + ``` + Example basic config, this will be sufficient for many users: ```yaml @@ -299,6 +323,22 @@ maps directly to this value. Follow existing Skaffold pattern - is there one? __Not Yet Resolved__ +6. Add a Google Cloud Build (`gcb`) support for the ko builder? + Other builders that support `gcb` have default public builder images. + The image `gcr.io/tekton-releases/ko-ci` is public, but do we want to + rely on it? Once ko is embedded in Skaffold, we could use + `gcr.io/k8s-skaffold/skaffold` as a default image.` + + __Not Yet Resolved__ + +7. Should we default dependency paths to `{"go.mod", "**.go"}` instead of + `{"."}`.? + + The former is a useful default for many (most?) Go apps, and it's used + in the `custom` example. The latter is the default for some other builders. + + __Not Yet Resolved__ + ## Implementation plan 1. Define interfaces in the ko codebase that allows ko to be used from From 1e46d57542498676d38f02bfe09989262e305ff3 Mon Sep 17 00:00:00 2001 From: Halvard Skogsrud Date: Wed, 21 Apr 2021 18:58:04 +1000 Subject: [PATCH 7/7] Ko builder proposal: Clarify proposed init behavior --- docs/design_proposals/ko-builder.md | 97 +++++++++++++++++------------ 1 file changed, 58 insertions(+), 39 deletions(-) diff --git a/docs/design_proposals/ko-builder.md b/docs/design_proposals/ko-builder.md index 435a7de89fb..aaa89ffa0fd 100644 --- a/docs/design_proposals/ko-builder.md +++ b/docs/design_proposals/ko-builder.md @@ -87,14 +87,7 @@ The ko builder supports and enhances these Skaffold Adding the ko builder requires making config changes to the Skaffold schema. -1. Add an entry to BuilderType enum in `proto/enums/enums.proto`: - - ```proto - // ko Builder - KO = 7; - ``` - -2. Add a `KoArtifact` type: +1. Add a `KoArtifact` type: ```go // KoArtifact builds images using [ko](https://github.com/google/ko). @@ -144,40 +137,32 @@ Adding the ko builder requires making config changes to the Skaffold schema. } ``` -3. Add a `KoArtifact` field to the `ArtifactType` struct: +2. Add a `KoArtifact` field to the `ArtifactType` struct: ```go type ArtifactType struct { + [...] // KoArtifact builds images using [ko](https://github.com/google/ko). KoArtifact *KoArtifact `yaml:"ko,omitempty" yamltags:"oneOf=artifact"` } ``` -4. Define ko's position in the initializer builder rank in - `pkg/skaffold/initializer/build/resolve.go`. The proposal is to add it - at the end, after the buildpacks builder: +3. Define `KoDependencies`: ```go - func builderRank(builder InitBuilder) int { - a := builder.ArtifactType("") - switch { - case a.DockerArtifact != nil: - return 1 - case a.JibArtifact != nil: - return 2 - case a.BazelArtifact != nil: - return 3 - case a.BuildpackArtifact != nil: - return 4 - case a.KoArtifact != nil: - return 5 - } - - return 6 + // KoDependencies is used to specify dependencies for an artifact built by ko. + type KoDependencies struct { + // Paths should be set to the file dependencies for this artifact, so that the skaffold file watcher knows when to rebuild and perform file synchronization. + // Defaults to {"go.mod", "**.go"} + Paths []string `yaml:"paths,omitempty" yamltags:"oneOf=dependency"` + + // Ignore specifies the paths that should be ignored by skaffold's file watcher. + // If a file exists in both `paths` and in `ignore`, it will be ignored, and will be excluded from both rebuilds and file synchronization. + Ignore []string `yaml:"ignore,omitempty"` } ``` -5. Add `KO` to the `BuilderType` enum in `proto/enums/enums.proto`: +4. Add `KO` to the `BuilderType` enum in `proto/enums/enums.proto`: ```proto enum BuilderType { @@ -200,10 +185,15 @@ Adding the ko builder requires making config changes to the Skaffold schema. } ``` +5. In `skaffold init`, default to the ko builder for any images where the + name starts with the ko prefix `ko://`. + +### Builder config schema + Example basic config, this will be sufficient for many users: ```yaml -apiVersion: skaffold/v2beta14 +apiVersion: skaffold/v2beta15 kind: Config build: artifacts: @@ -217,7 +207,7 @@ The value of the `image` field is the Go import path of the app entry point, A more comprehensive example config: ```yaml -apiVersion: skaffold/v2beta14 +apiVersion: skaffold/v2beta15 kind: Config build: artifacts: @@ -331,7 +321,15 @@ maps directly to this value. __Not Yet Resolved__ -7. Should we default dependency paths to `{"go.mod", "**.go"}` instead of +7. File sync support: Should we limit this to + [ko static assets](https://github.com/google/ko#static-assets) only? + + This is the only way to include additional files in a container image + built by ko. + + __Not Yet Resolved__ + +8. Should we default dependency paths to `{"go.mod", "**.go"}` instead of `{"."}`.? The former is a useful default for many (most?) Go apps, and it's used @@ -341,14 +339,33 @@ maps directly to this value. ## Implementation plan -1. Define interfaces in the ko codebase that allows ko to be used from +1. Define integration points in the ko codebase that allows ko to be used from Skaffold without duplicating existing ko CLI code. - Draft interface: + In the package `github.com/google/ko/pkg/commands`: ```go - type KoPublish interface { - func BuildAndPublish(context.Context, options.BuildOptions, options.PublishOptions, importpath string, imageNameWithTag string) + // SetDefaultBaseImage enables programmatically overriding the base image, + // as an alternative to specifying it in a `.ko.yaml` file. + func SetDefaultBaseImage(baseImage string) error // maps to the fromImage option + + var UserAgent func () string // allow overriding with the Skaffold user agent + + // MakeBuilder creates a build.Interface, delegates to the existing makeBuilder() + func MakeBuilder(ctx context.Context, bo *options.BuildOptions) (build.Interface, error) + + // MakePublisher creates a publish.Interface, delegates to the existing makePublisher() + func MakePublisher(po *options.PublishOptions) (publish.Interface, error) + ``` + + In the package `github.com/google/ko/pkg/commands/options`, allow + specifying the default repo as a flag: + + ```go + type PublishOptions struct { + // DockerRepo overrides the KO_DOCKER_REPO environment variable, if present + DockerRepo string + [...] } ``` @@ -368,7 +385,7 @@ maps directly to this value. Example `skaffold.yaml` supported at this stage: ```yaml - apiVersion: skaffold/v2beta14 + apiVersion: skaffold/v2beta15 kind: Config build: artifacts: @@ -386,7 +403,6 @@ maps directly to this value. platforms: - linux/amd64 - linux/arm64 - sourceDateEpoch: 946684800 ``` 3. Implement support for additional config options in ko: @@ -418,4 +434,7 @@ Please describe what new test cases you are going to consider. [`--disable-optimization`](https://github.com/google/ko/blob/f7df8106196518df5c6c35432843421e33990329/pkg/commands/options/build.go#L34) is added for debugging. -3. Add ko example to the `examples` directory. +3. Add basic and comprehensive ko examples to the `integration/examples` + directory. + +4. TBC