diff --git a/docs/design_proposals/ko-builder.md b/docs/design_proposals/ko-builder.md index ecde4d4fd30..effbf7bd91a 100644 --- a/docs/design_proposals/ko-builder.md +++ b/docs/design_proposals/ko-builder.md @@ -83,6 +83,135 @@ The ko builder supports and enhances these Skaffold installing additional tools or keeping toolchain versions in sync across local development and CI/CD. +## Background: ko image names and Go import paths + +Ko uses Go import paths to build images. The +[`ko publish`](https://github.com/google/ko#build-an-image) command takes a +required positional argument, which can be either a local file path or a Go +import path. If the argument is a local file path (as per +[`go/build.IsLocalImport()`](https://pkg.go.dev/go/build#IsLocalImport)) +then, ko resolves the local file path to a Go import path (see +[`github.com/google/ko/pkg/build`](https://github.com/google/ko/blob/ab4d264103bd4931c6721d52bfc9d1a2e79c81d1/pkg/build/gobuild.go#L261)). + +The import path must be of the package than contains the `main()` function. +For instance, to build Skaffold using ko, from the repository root directory: + +```sh +ko publish ./cmd/skaffold +``` + +or + +```sh +ko publish github.com/GoogleContainerTools/skaffold/cmd/skaffold +``` + +When the ko CLI is used to +[populate the image name in templated Kubernetes resource files](https://github.com/google/ko#kubernetes-integration), +only the Go import path option can be used, and the import path must be +prefixed by the `ko://` scheme, e.g., +`ko://github.com/GoogleContainerTools/skaffold/cmd/skaffold`. + +Ko determines the image name from the container image registry (provided by the +`KO_DOCKER_REPO` environment variable) and the Go import path. The Go import +path is appended in one of these ways: + +- The last path segment (e.g., `skaffold`), followed by a hyphen and a MD5 + hash. This is the default behavior of the `ko publish` command. + +- The last path segment (e.g., `skaffold`) only, if `ko publish` is invoked + with the `-B` or `--base-import-paths` flag. + +- The full import path, lowercased (e.g., + `github.com/googlecontainertools/skaffold/cmd/skaffold`), if `ko publish` is + invoked with the `-P` or `--preserve-import-paths` flag. This is the option + used by projects such as Knative (see the + [`release.sh` script](https://github.com/knative/serving/blob/v0.24.0/vendor/knative.dev/hack/release.sh#L98)) + and Tekton + (see the pipeline in + [publish.yaml](https://github.com/tektoncd/pipeline/blob/v0.25.0/tekton/publish.yaml#L137)). + +- No import path (just `KO_DOCKER_REPO`), if `ko publish` is invoked with the + `--bare` flag. + +## Supporting existing Skaffold users + +The Skaffold ko builder follows the existing Skaffold image naming logic. This +means that the image naming behavior doesn't change for existing Skaffold users +who migrate from other builders to the ko builder. + +The ko builder achieves this by using ko's +[`Bare`](https://github.com/google/ko/blob/ab4d264103bd4931c6721d52bfc9d1a2e79c81d1/pkg/commands/options/publish.go#L60) +naming option. + +By using this option, the image name is not tied to the Go import path. If the +Skaffold +[default repo](https://skaffold.dev/docs/environment/image-registries/) value +is `gcr.io/k8s-skaffold` and tne value of the `image` field in `skaffold.yaml` +is `skaffold`, the resulting image name will be `gcr.io/k8s-skaffold/skaffold`. + +It is still necessary to resolve the Go import path for the underlying ko +implementation. To do so, the ko builder determines the import path of the +current +[`context`](https://skaffold.dev/docs/references/yaml/#build-artifacts-context) +(a.k.a. +[`Workspace`](https://github.com/GoogleContainerTools/skaffold/blob/v1.27.0/pkg/skaffold/schema/latest/v1/config.go#L832)) +directory. + +By specifying different `context` directories for each `artifact` in +`skaffold.yaml`, the ko builder supports building multiple artifacts in the +same Skaffold config, such as in the +[microservices example](https://github.com/GoogleContainerTools/skaffold/tree/v1.27.0/examples/microservices). + +## Supporting existing ko users + +To support existing ko users moving to Skaffold, the ko builder also supports +`image` names in `skaffold.yaml` that use the Go import path, prefixed by the +`ko://` scheme. Examples of such image references in Kubernetes manifest files +can be seen in projects such as +[Knative](https://github.com/knative/serving/blob/main/config/core/deployments/activator.yaml#L41) +and +[Tekton](https://github.com/tektoncd/pipeline/blob/v0.25.0/config/controller.yaml#L66). + +In the case of `ko://`-prefixed image names, the Skaffold ko builder +constructs the image name by: + +1. Removing the `ko://` scheme prefix. +2. Transforming the import path to a valid image name using the function + [`SanitizeImageName()`](https://github.com/GoogleContainerTools/skaffold/blob/v1.27.0/pkg/skaffold/docker/reference.go#L83) + (from the package + `github.com/GoogleContainerTools/skaffold/pkg/skaffold/docker`). +3. Combining the Skaffold default repo with the transformed import path as per + existing Skaffold image naming logic. + +This will result in image names that match those produced by the `ko` CLI when +using the `-P` or `--preserve-import-paths` flag. For example, if the Skaffold +default repo is `gcr.io/k8s-skaffold` and the `image` name in `skaffold.yaml` +is `ko://github.com/GoogleContainerTools/skaffold/cmd/skaffold`, the resulting +image name will be +`gcr.io/k8s-skaffold/github.com/googlecontainertools/skaffold/cmd/skaffold`. + +Real-world examples of image names that follow this naming convention can be +found in the Tekton and Knative release manifests. For instance, view the +images in the Knative Serving release YAMLs: + +```sh +curl -sL https://github.com/knative/serving/releases/download/v0.24.0/serving-core.yaml | grep 'image: ' +``` + +If the `image` field in `skaffold.yaml` starts with the `ko://` scheme prefix, +the Skaffold ko builder uses the Go import path that follows the prefix. If the +`image` name in `skaffold.yaml` does _not_ start with `ko://`, then the ko +builder determines the Go import path from the artifact `context` directory. + +Users who want to build an artifact where the `main()` function is _not_ in the +`context` directory must specify the full import path in the image name. For +instance, to build Skaffold itself using the Skaffold ko builder, for a +`context` directory of `.` (the default), the `image` name must be +`ko://github.com/GoogleContainerTools/skaffold/cmd/skaffold`. +Image names that start with relative path references such as `./cmd/skaffold` +are _not_ supported by Skaffold. + ## Design Adding the ko builder requires making config changes to the Skaffold schema. @@ -196,11 +325,11 @@ Adding the ko builder requires making config changes to the Skaffold schema. Example basic config, this will be sufficient for many users: ```yaml -apiVersion: skaffold/v2beta15 +apiVersion: skaffold/v2beta19 kind: Config build: artifacts: - - image: ko://github.com/GoogleContainerTools/skaffold/examples/ko + - image: skaffold-example-ko ko: {} ``` @@ -210,7 +339,7 @@ The value of the `image` field is the Go import path of the app entry point, A more comprehensive example config: ```yaml -apiVersion: skaffold/v2beta15 +apiVersion: skaffold/v2beta19 kind: Config build: artifacts: @@ -313,8 +442,6 @@ maps directly to this value. and . - - ### Open questions 1. Should we default dependency paths to `{"go.mod", "**.go"}` instead of @@ -327,10 +454,8 @@ maps directly to this value. 2. Add a Google Cloud Build (`gcb`) support for the ko builder? - Other builders that support `gcb` have default public builder images. - The image `gcr.io/tekton-releases/ko-ci` is public, but do we want to - rely on it? Once ko is embedded in Skaffold, we could use - `gcr.io/k8s-skaffold/skaffold` as a default image.` + By embedding ko as a module, there is no need for a ko-specific Skaffold + builder image. __Not Yet Resolved__ @@ -347,6 +472,7 @@ maps directly to this value. file? Suggest yes, to make Skaffold a compelling choice for Go developers. + __Not Yet Resolved__ ## Approach @@ -479,13 +605,13 @@ The steps roughly outlined: Example `skaffold.yaml` supported at this stage: ```yaml - apiVersion: skaffold/v2beta18 + apiVersion: skaffold/v2beta19 kind: Config build: artifacts: - image: skaffold-ko ko: - fromImage: gcr.io/distroless/static-debian10:nonroot + fromImage: gcr.io/distroless/base:nonroot dependencies: paths: - go.mod @@ -498,8 +624,8 @@ The steps roughly outlined: - linux/arm64 ``` -3. After [google/ko#340](https://github.com/google/ko/pull/340) is merged, - implement Skaffold config support for additional ko config options: +3. Implement Skaffold config support for additional ko config options added in + [google/ko#340](https://github.com/google/ko/pull/340): - `args`, e.g., `-v`, `-trimpath` - `asmflags` diff --git a/go.mod b/go.mod index ec471be99d7..88127dd3d84 100644 --- a/go.mod +++ b/go.mod @@ -41,7 +41,7 @@ require ( github.com/google/go-containerregistry v0.5.1 github.com/google/go-containerregistry/pkg/authn/k8schain v0.0.0-20210216200643-d81088d9983e // indirect github.com/google/go-github v17.0.0+incompatible - github.com/google/ko v0.8.4-0.20210615195035-ee2353837872 + github.com/google/ko v0.8.4-0.20210715141624-56282bf645ea github.com/google/uuid v1.1.2 github.com/grpc-ecosystem/grpc-gateway v1.14.8 github.com/heroku/color v0.0.6 diff --git a/go.sum b/go.sum index 790a51979dd..46f478b4645 100644 --- a/go.sum +++ b/go.sum @@ -193,6 +193,7 @@ github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRF github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d h1:UQZhZ2O0vMHr2cI+DC1Mbh0TJxzA3RcLoMsFw+aXw7E= github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d/go.mod h1:rBZYJk541a8SKzHPHnH3zbiI+7dagKZ0cgpgrD7Fyho= +github.com/alessio/shellescape v1.2.2 h1:8LnL+ncxhWT2TR00dfJRT25JWWrhkMZXneHVWnetDZg= github.com/alessio/shellescape v1.2.2/go.mod h1:PZAiSCk0LJaZkiCSkPv8qIobYglO3FPpyFjDCtHLS30= github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883/go.mod h1:rCTlJbsFo29Kk6CurOXKm700vrz8f0KW0JNfpkRJY/8= github.com/andybalholm/cascadia v1.0.0/go.mod h1:GsXiBklL0woXo1j/WYWtSYYC4ouU9PqHO0sqidkEA4Y= @@ -419,6 +420,7 @@ github.com/docker/libtrust v0.0.0-20150114040149-fa567046d9b1/go.mod h1:cyGadeNE github.com/docker/spdystream v0.0.0-20160310174837-449fdfce4d96 h1:cenwrSVm+Z7QLSV/BsnenAOcDXdX4cMv4wP0B/5QbPg= github.com/docker/spdystream v0.0.0-20160310174837-449fdfce4d96/go.mod h1:Qh8CwZgvJUkLughtfhJv5dyTYa91l1fOUCrgjqmcifM= github.com/docopt/docopt-go v0.0.0-20180111231733-ee0de3bc6815/go.mod h1:WwZ+bS3ebgob9U8Nd0kOddGdZWjyMGR8Wziv+TBNwSE= +github.com/dprotaso/go-yit v0.0.0-20191028211022-135eb7262960 h1:aRd8M7HJVZOqn/vhOzrGcQH0lNAMkqMn+pXUYkatmcA= github.com/dprotaso/go-yit v0.0.0-20191028211022-135eb7262960/go.mod h1:9HQzr9D/0PGwMEbC3d5AB7oi67+h4TsQqItC1GVYG58= github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk= github.com/dustin/go-humanize v1.0.0 h1:VSnTsYCnlFHaM2/igO1h6X3HA71jcobQuxemgkq4zYo= @@ -432,6 +434,7 @@ github.com/elazarl/goproxy v0.0.0-20170405201442-c4fc26588b6e/go.mod h1:/Zj4wYkg github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153 h1:yUdfgN0XgIJw7foRItutHYUIhlcKzcSf5vDpdhQAKTc= github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153/go.mod h1:/Zj4wYkgs4iZTTu3o/KG3Itv/qCCa8VVMlb3i9OVuzc= github.com/emicklei/go-restful v0.0.0-20170410110728-ff4f55a20633/go.mod h1:otzb+WCGbkyDHkqmQmT5YD2WR4BBwUdeQoFo8l/7tVs= +github.com/emicklei/go-restful v2.9.5+incompatible h1:spTtZBk5DYEvbxMVutUuTyh1Ao2r4iyvLdACqsl/Ljk= github.com/emicklei/go-restful v2.9.5+incompatible/go.mod h1:otzb+WCGbkyDHkqmQmT5YD2WR4BBwUdeQoFo8l/7tVs= github.com/emirpasic/gods v1.12.0 h1:QAUIPSaCu4G+POclxeqb3F+WPpdKqFGlw36+yOzGlrg= github.com/emirpasic/gods v1.12.0/go.mod h1:YfzfFFoVP/catgzJb4IKIqXjX78Ha8FMSDh3ymbK86o= @@ -448,6 +451,7 @@ github.com/evanphx/json-patch v4.2.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLi github.com/evanphx/json-patch v4.5.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk= github.com/evanphx/json-patch v4.9.0+incompatible h1:kLcOMZeuLAJvL2BPWLMIj5oaZQobrkAqrL+WFZwQses= github.com/evanphx/json-patch v4.9.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk= +github.com/evanphx/json-patch/v5 v5.0.0 h1:dKTrUeykyQwKb/kx7Z+4ukDs6l+4L41HqG1XHnhX7WE= github.com/evanphx/json-patch/v5 v5.0.0/go.mod h1:G79N1coSVB93tBe7j6PhzjmR3/2VvlbKOFpnXhI9Bw4= github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d/go.mod h1:ZZMPRZwes7CROmyNKgQzC3XPs6L/G2EJLHddWejkmf4= github.com/fatih/camelcase v1.0.0/go.mod h1:yN2Sb0lFhZJUdVvtELVWefmrXpuZESvPmqwoZc+/fpc= @@ -666,6 +670,7 @@ github.com/gonum/matrix v0.0.0-20181209220409-c518dec07be9/go.mod h1:0EXg4mc1CNP github.com/gonum/stat v0.0.0-20181125101827-41a0da705a5b/go.mod h1:Z4GIJBJO3Wa4gD4vbwQxXXZ+WHmW6E9ixmNrwvs0iZs= github.com/google/btree v0.0.0-20180124185431-e89373fe6b4a/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= +github.com/google/btree v1.0.0 h1:0udJVsspx3VBr5FwtLhQQtuAsVc79tTq0ocGIPAU6qo= github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= github.com/google/crfs v0.0.0-20191108021818-71d77da419c9/go.mod h1:etGhoOqfwPkooV6aqoX3eBGQOJblqdoc9XvWOeuxpPw= github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M= @@ -709,6 +714,8 @@ github.com/google/gofuzz v1.1.0 h1:Hsa8mG0dQ46ij8Sl2AYJDUv1oA9/d6Vk+3LG99Oe02g= github.com/google/gofuzz v1.1.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/google/ko v0.8.4-0.20210615195035-ee2353837872 h1:FCOz4UI3A9VlHpWI+6Kx4njvD3HOxy5TG+5K/2OJ/Fo= github.com/google/ko v0.8.4-0.20210615195035-ee2353837872/go.mod h1:4k+PvVaGdNPZZqUyrwJDbq+BrCdj1PM+jspkOQGDwNA= +github.com/google/ko v0.8.4-0.20210715141624-56282bf645ea h1:wUf6x+jugp/EgKi3FObQuy4uxRVvoAwjilyhwml17JY= +github.com/google/ko v0.8.4-0.20210715141624-56282bf645ea/go.mod h1:4k+PvVaGdNPZZqUyrwJDbq+BrCdj1PM+jspkOQGDwNA= github.com/google/mako v0.0.0-20190821191249-122f8dcef9e3/go.mod h1:YzLcVlL+NqWnmUEPuhS1LxDDwGO9WNbVlEXaF4IH35g= github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs= github.com/google/martian v2.1.1-0.20190517191504-25dcb96d9e51+incompatible h1:xmapqc1AyLoB+ddYT6r04bD9lIjlOqGaREovi0SzFaE= @@ -753,6 +760,7 @@ github.com/googleinterns/cloud-operations-api-mock v0.0.0-20200709193332-a1e58c2 github.com/googleinterns/cloud-operations-api-mock v0.0.0-20200709193332-a1e58c29bdd3/go.mod h1:h/KNeRx7oYU4SpA4SoY7W2/NxDKEEVuwA6j9A27L4OI= github.com/gookit/color v1.2.4/go.mod h1:AhIE+pS6D4Ql0SQWbBeXPHw7gY0/sjHoA4s/n1KB7xg= github.com/gophercloud/gophercloud v0.1.0/go.mod h1:vxM41WHh5uqHVBMZHzuwNOHh8XEoIEcSTewFxm1c5g8= +github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1 h1:EGx4pi6eqNxGaHF6qqu48+N2wcFQ5qg5FXgOdqsJ5d8= github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY= github.com/goreleaser/goreleaser v0.136.0/go.mod h1:wiKrPUeSNh6Wu8nUHxZydSOVQ/OZvOaO7DTtFqie904= github.com/goreleaser/nfpm v1.2.1/go.mod h1:TtWrABZozuLOttX2uDlYyECfQX7x5XYkVxhjYcR6G9w= @@ -772,6 +780,7 @@ github.com/gostaticanalysis/analysisutil v0.0.0-20190318220348-4088753ea4d3/go.m github.com/gostaticanalysis/analysisutil v0.0.3/go.mod h1:eEOZF4jCKGi+aprrirO9e7WKB3beBRtWgqGunKl6pKE= github.com/gotestyourself/gotestyourself v2.2.0+incompatible/go.mod h1:zZKM6oeNM8k+FRljX1mnzVYeS8wiGgQyvST1/GafPbY= github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7/go.mod h1:FecbI9+v66THATjSRHfNgh1IVFe/9kFxbXtjV0ctIMA= +github.com/gregjones/httpcache v0.0.0-20190212212710-3befbb6ad0cc h1:f8eY6cV/x1x+HLjOp4r72s/31/V2aTUtg5oKRRPf8/Q= github.com/gregjones/httpcache v0.0.0-20190212212710-3befbb6ad0cc/go.mod h1:FecbI9+v66THATjSRHfNgh1IVFe/9kFxbXtjV0ctIMA= github.com/grpc-ecosystem/go-grpc-middleware v1.0.0/go.mod h1:FiyG127CGDf3tlThmgyCl78X/SZQqEOJBCDaAfeWzPs= github.com/grpc-ecosystem/go-grpc-middleware v1.0.1-0.20190118093823-f849b5445de4/go.mod h1:FiyG127CGDf3tlThmgyCl78X/SZQqEOJBCDaAfeWzPs= @@ -815,6 +824,7 @@ github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ github.com/hashicorp/golang-lru v0.5.3/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4= github.com/hashicorp/golang-lru v0.5.4 h1:YDjusn29QI/Das2iO9M0BHnIbxPeyuCHsjMW+lJfyTc= github.com/hashicorp/golang-lru v0.5.4/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4= +github.com/hashicorp/hcl v1.0.0 h1:0Anlzjpi4vEasTeNFn2mLJgTSwt0+6sfsiTG8qcWGx4= github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ= github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64= github.com/hashicorp/mdns v1.0.0/go.mod h1:tL+uN++7HEJ6SQLQ2/p+z2pH24WQKWjBPkE0mNTz8vQ= @@ -872,6 +882,7 @@ github.com/json-iterator/go v1.1.10/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/ github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU= github.com/jstemmer/go-junit-report v0.9.1 h1:6QPYqodiu3GuPL+7mfx+NwDdp2eTkp9IfEUpgAwUN0o= github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk= +github.com/jtolds/gls v4.20.0+incompatible h1:xdiiI2gbIgH/gLH7ADydsJ1uDOEzR8yvV7C0MuV77Wo= github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU= github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w= github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM= @@ -914,6 +925,7 @@ github.com/kylelemons/godebug v0.0.0-20170820004349-d65d576e9348/go.mod h1:B69LE github.com/lib/pq v1.0.0/go.mod h1:5WUZQaWbwv1U+lTReE5YruASi9Al49XbQIvNi/34Woo= github.com/lib/pq v1.1.1/go.mod h1:5WUZQaWbwv1U+lTReE5YruASi9Al49XbQIvNi/34Woo= github.com/lib/pq v1.2.0/go.mod h1:5WUZQaWbwv1U+lTReE5YruASi9Al49XbQIvNi/34Woo= +github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de h1:9TO3cAIGXtEhnIaL+V+BEER86oLrvS+kWobKpbJuye0= github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de/go.mod h1:zAbeS9B/r2mtpb6U+EI2rYA5OAXxsYw6wTamcNW+zcE= github.com/lightstep/lightstep-tracer-common/golang/gogo v0.0.0-20190605223551-bc2310a04743/go.mod h1:qklhhLq1aX+mtWk9cPHPzaBjWImj5ULL6C7HFJtXQMM= github.com/lightstep/lightstep-tracer-go v0.18.1/go.mod h1:jlF1pusYV4pidLvZ+XD0UBX0ZE6WURAspgAczcDHrL4= @@ -921,6 +933,7 @@ github.com/lithammer/dedent v1.1.0/go.mod h1:jrXYCQtgg0nJiN+StA2KgR7w6CiQNv9Fd/Z github.com/logrusorgru/aurora v0.0.0-20181002194514-a7b3b318ed4e/go.mod h1:7rIyQOR62GCctdiQpZ/zOJlFyk6y+94wXzv6RNZgaR4= github.com/lyft/protoc-gen-validate v0.0.13/go.mod h1:XbGvPuh87YZc5TdIa2/I4pLk0QoUACkjt2znoq26NVQ= github.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ= +github.com/magiconair/properties v1.8.1 h1:ZC2Vc7/ZFkGmsVC9KvOjumD+G5lXy2RtTKyzRKO2BQ4= github.com/magiconair/properties v1.8.1/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ= github.com/mailru/easyjson v0.0.0-20160728113105-d5b7844b561a/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc= github.com/mailru/easyjson v0.0.0-20180823135443-60711f1a8329/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc= @@ -934,6 +947,7 @@ github.com/maratori/testpackage v1.0.1/go.mod h1:ddKdw+XG0Phzhx8BFDTKgpWP4i7MpAp github.com/markbates/inflect v1.0.4/go.mod h1:1fR9+pO2KHEO9ZRtto13gDwwZaAKstQzferVeWqbgNs= github.com/marstr/guid v1.1.0/go.mod h1:74gB1z2wpxxInTG6yaqA7KrtM0NZ+RbrcqDvYHefzho= github.com/matoous/godox v0.0.0-20190911065817-5d6d842e92eb/go.mod h1:1BELzlh859Sh1c6+90blK8lbYy0kwQf1bYlBhBysy1s= +github.com/mattmoor/dep-notify v0.0.0-20190205035814-a45dec370a17 h1:P91eDVgVzvF2EmA6fmGCyR2VQFlmo2nsmS2DbHoGAco= github.com/mattmoor/dep-notify v0.0.0-20190205035814-a45dec370a17/go.mod h1:qWnF4u+oS4UWOZMwZcBQXrt5IQIdWc6XVJLDdxGIfdQ= github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU= github.com/mattn/go-colorable v0.1.1/go.mod h1:FuOcm+DKB9mbwrcAfNl7/TZVBZ6rcnceauSikq3lYCQ= @@ -980,6 +994,7 @@ github.com/mitchellh/ioprogress v0.0.0-20180201004757-6a23b12fa88e h1:Qa6dnn8Dla github.com/mitchellh/ioprogress v0.0.0-20180201004757-6a23b12fa88e/go.mod h1:waEya8ee1Ro/lgxpVhkJI4BVASzkm3UZqkx/cFJiYHM= github.com/mitchellh/mapstructure v0.0.0-20160808181253-ca63d7c062ee/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y= github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y= +github.com/mitchellh/mapstructure v1.3.1 h1:cCBH2gTD2K0OtLlv/Y5H01VQCqmlDxz30kS5Y5bqfLA= github.com/mitchellh/mapstructure v1.3.1/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo= github.com/mitchellh/osext v0.0.0-20151018003038-5e2d6d41470f/go.mod h1:OkQIRizQZAeMln+1tSwduZz7+Af5oFlKirV/MSYes2A= github.com/moby/buildkit v0.8.0 h1:isPRu9bp8xbMSvs8P6yHAc8vsYPFVNFKrOXHe/tzNXw= @@ -1104,6 +1119,7 @@ github.com/pelletier/go-toml v1.8.0/go.mod h1:D6yutnOGMveHEPV7VQOuvI/gXY61bv+9bA github.com/pelletier/go-toml v1.9.0 h1:NOd0BRdOKpPf0SxkL3HxSQOG7rNh+4kl6PHcBPFs7Q0= github.com/pelletier/go-toml v1.9.0/go.mod h1:u1nR/EPcESfeI/szUZKdtJ0xRNbUoANCkoOuaOx1Y+c= github.com/performancecopilot/speed v3.0.0+incompatible/go.mod h1:/CLtqpZ5gBg1M9iaPbIdPPGyKcA8hKdoy6hAWba7Yac= +github.com/peterbourgon/diskv v2.0.1+incompatible h1:UBdAOUP5p4RWqPBg048CAvpKN+vxiaj6gdUUzhl4XmI= github.com/peterbourgon/diskv v2.0.1+incompatible/go.mod h1:uqqh8zWWbv1HBMNONnaR/tNboyR3/BZd58JJSHlUSCU= github.com/phayes/checkstyle v0.0.0-20170904204023-bfd46e6a821d/go.mod h1:3OzsM7FXDQlpCiw2j81fOmAwQLnZnLGXVKUzeKQXIAw= github.com/pierrec/lz4 v1.0.2-0.20190131084431-473cd7ce01a1/go.mod h1:3/3N9NVKO0jef7pBehbT1qWhCMrIgbYNnFAZCqQ5LRc= @@ -1231,9 +1247,11 @@ github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic github.com/sirupsen/logrus v1.8.1 h1:dJKuHgqk1NNQlqoA6BTlM1Wf9DOH3NBjQyu0h9+AZZE= github.com/sirupsen/logrus v1.8.1/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0= github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc= +github.com/smartystreets/assertions v1.0.0 h1:UVQPSSmc3qtTi+zPPkCXvZX9VvW/xT/NsRvKfwY81a8= github.com/smartystreets/assertions v1.0.0/go.mod h1:kHHU4qYBaI3q23Pp3VPrmWhuIUrLW/7eUrw0BU5VaoM= github.com/smartystreets/go-aws-auth v0.0.0-20180515143844-0c1422d1fdb9/go.mod h1:SnhjPscd9TpLiy1LpzGSKh3bXCfxxXuqd9xmQJy3slM= github.com/smartystreets/goconvey v0.0.0-20190330032615-68dc04aab96a/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA= +github.com/smartystreets/goconvey v1.6.4 h1:fv0U8FUIMPNf1L9lnHLvLhgicrIVChEkdzIKYqbNC9s= github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA= github.com/smartystreets/gunit v1.0.0/go.mod h1:qwPWnhz6pn0NnRBP++URONOVyNkPyr4SauJk4cUOwJs= github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4kGIyLM= @@ -1242,8 +1260,10 @@ github.com/sourcegraph/go-diff v0.5.1/go.mod h1:j2dHj3m8aZgQO8lMTcTnBcXkRRRqi34c github.com/sourcegraph/go-diff v0.5.3/go.mod h1:v9JDtjCE4HHHCZGId75rg8gkKKa98RVjBcBGsVmMmak= github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA= github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ= +github.com/spf13/afero v1.2.2 h1:5jhuqJyZCZf2JRofRvN/nIFgIWNzPa3/Vz8mYylgbWc= github.com/spf13/afero v1.2.2/go.mod h1:9ZxEEn6pIJ8Rxe320qSDBk6AsU0r9pR7Q4OcevTdifk= github.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE= +github.com/spf13/cast v1.3.1 h1:nFm6S0SMdyzrzcmThSipiEubIDy8WEXKNZ0UOgiRpng= github.com/spf13/cast v1.3.1/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE= github.com/spf13/cobra v0.0.2-0.20171109065643-2da4a54c5cee/go.mod h1:1l0Ry5zgKvJasoi3XT1TypsSe7PqH0Sj9dhYf7v3XqQ= github.com/spf13/cobra v0.0.3/go.mod h1:1l0Ry5zgKvJasoi3XT1TypsSe7PqH0Sj9dhYf7v3XqQ= @@ -1252,6 +1272,7 @@ github.com/spf13/cobra v1.0.0/go.mod h1:/6GTrnGXV9HjY+aR4k0oJ5tcvakLuG6EuKReYlHN github.com/spf13/cobra v1.1.3 h1:xghbfqPkxzxP3C/f3n5DdpAbdKLj4ZE4BWQI362l53M= github.com/spf13/cobra v1.1.3/go.mod h1:pGADOWyqRD/YMrPZigI/zbliZ2wVD/23d+is3pSWzOo= github.com/spf13/jwalterweatherman v1.0.0/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo= +github.com/spf13/jwalterweatherman v1.1.0 h1:ue6voC5bR5F8YxI5S67j9i582FU4Qvo2bmqnqMYADFk= github.com/spf13/jwalterweatherman v1.1.0/go.mod h1:aNWZUN0dPAAO/Ljvb5BEdw96iTZ0EXowPYD95IqWIGo= github.com/spf13/pflag v0.0.0-20170130214245-9ff6c6923cff/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4= github.com/spf13/pflag v1.0.1-0.20171106142849-4c012f6dcd95/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4= @@ -1262,6 +1283,7 @@ github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An github.com/spf13/viper v1.3.2/go.mod h1:ZiWeW+zYFKm7srdB9IoDzzZXaJaI5eL9QjNiN/DMA2s= github.com/spf13/viper v1.4.0/go.mod h1:PTJ7Z/lr49W6bUbkmS1V3by4uWynFiR9p7+dSq/yZzE= github.com/spf13/viper v1.6.1/go.mod h1:t3iDnF5Jlj76alVNuyFBk5oUMCvsrkbvZK0WQdfDi5k= +github.com/spf13/viper v1.7.0 h1:xVKxvI7ouOI5I+U9s2eeiUfMaWBVoXA3AWskkrqK0VM= github.com/spf13/viper v1.7.0/go.mod h1:8WkrPz2fc9jxqZNCJI/76HCieCp4Q8HaLFoCha5qpdg= github.com/src-d/gcfg v1.4.0 h1:xXbNR5AlLSA315x2UO+fTSSAXCDf+Ar38/6oyGbDKQ4= github.com/src-d/gcfg v1.4.0/go.mod h1:p/UMsR43ujA89BJY9duynAwIpvqEujIH/jFlfL7jWoI= @@ -1285,6 +1307,7 @@ github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5 github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY= github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= +github.com/subosito/gotenv v1.2.0 h1:Slr1R9HxAlEKefgq5jn9U+DnETlIUa6HfgEzj0g5d7s= github.com/subosito/gotenv v1.2.0/go.mod h1:N0PQaV/YGNqwC0u51sEeR/aUtSLEXKX9iv69rRypqCw= github.com/syndtr/gocapability v0.0.0-20170704070218-db04d3cc01c8/go.mod h1:hkRG7XYTFWNJGYcbNJQlaLq0fg1yr4J4t/NcTQtrfww= github.com/syndtr/gocapability v0.0.0-20180916011248-d98352740cb2/go.mod h1:hkRG7XYTFWNJGYcbNJQlaLq0fg1yr4J4t/NcTQtrfww= @@ -2006,6 +2029,7 @@ gopkg.in/gemnasium/logrus-airbrake-hook.v2 v2.1.2/go.mod h1:Xk6kEKp8OKb+X14hQBKW gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc= gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw= gopkg.in/ini.v1 v1.51.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k= +gopkg.in/ini.v1 v1.56.0 h1:DPMeDvGTM54DXbPkVIZsp19fp/I2K7zwA/itHYHKo8Y= gopkg.in/ini.v1 v1.56.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k= gopkg.in/natefinch/lumberjack.v2 v2.0.0/go.mod h1:l0ndWWf7gzL7RNwBG7wST/UCcT4T24xpD6X8LsfU/+k= gopkg.in/resty.v1 v1.12.0/go.mod h1:mDo4pnntr5jdWRML875a/NmxYqAlA73dVijT2AXvQQo= @@ -2075,6 +2099,7 @@ k8s.io/apiserver v0.18.8/go.mod h1:12u5FuGql8Cc497ORNj79rhPdiXQC4bf53X/skR/1YM= k8s.io/apiserver v0.18.12/go.mod h1:uFOeW4LlxS6KDgLWy3n3gh0DhC6m41QIFgL33ouk+4w= k8s.io/apiserver v0.19.7/go.mod h1:DmWVQggNePspa+vSsVytVbS3iBSDTXdJVt0akfHacKk= k8s.io/cli-runtime v0.19.4/go.mod h1:m8G32dVbKOeaX1foGhleLEvNd6REvU7YnZyWn5//9rw= +k8s.io/cli-runtime v0.19.7 h1:VkHsqrQYCD6+yBm2k9lOxLJtfo1tmb/TdYIHQ2RSCsY= k8s.io/cli-runtime v0.19.7/go.mod h1:UTtbWaGV/USZSrnvuW/lRZGM5OsemAT/q/Du/Ac+wKU= k8s.io/client-go v0.0.0-20180910083459-2cefa64ff137/go.mod h1:7vJpHMYJwNQCWgzmNV+VYUl1zCObLyodBc8nIyt8L5s= k8s.io/client-go v0.17.4/go.mod h1:ouF6o5pz3is8qU0/qYL2RnoxOPqgfuidYLowytyLJmc= @@ -2160,6 +2185,7 @@ rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0= rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA= sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.0.7/go.mod h1:PHgbrJT7lCHcxMU+mDHEm+nx46H4zuuHZkDP6icnhu0= sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.0.9/go.mod h1:dzAXnQbTRyDlZPJX2SUPEqvnB+j7AJjtlox7PEwigU0= +sigs.k8s.io/kind v0.8.1 h1:9wsEbEtMQV9QObaqS/T4VxBeXXPtu+qM9sFMqgO/90o= sigs.k8s.io/kind v0.8.1/go.mod h1:oNKTxUVPYkV9lWzY6CVMNluVq8cBsyq+UgPJdvA3uu4= sigs.k8s.io/kustomize v2.0.3+incompatible h1:JUufWFNlI44MdtnjUqVnvh29rR37PQFzPbLXqhyOyX0= sigs.k8s.io/kustomize v2.0.3+incompatible/go.mod h1:MkjgH3RdOWrievjo6c9T245dYlB5QeXV4WCbnt/PEpU= diff --git a/pkg/skaffold/build/ko/build.go b/pkg/skaffold/build/ko/build.go new file mode 100644 index 00000000000..dc5033dafb1 --- /dev/null +++ b/pkg/skaffold/build/ko/build.go @@ -0,0 +1,100 @@ +/* +Copyright 2021 The Skaffold Authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package ko + +import ( + "context" + "fmt" + "io" + "strings" + + "github.com/google/go-containerregistry/pkg/name" + "github.com/google/ko/pkg/build" + "github.com/google/ko/pkg/publish" + + // latestV1 "github.com/GoogleContainerTools/skaffold/pkg/skaffold/schema/latest/v1" + latestV1 "github.com/GoogleContainerTools/skaffold/pkg/skaffold/build/ko/schema" +) + +// Build an artifact using ko +func (b *Builder) Build(ctx context.Context, out io.Writer, a *latestV1.Artifact, ref string) (string, error) { + koBuilder, err := b.newKoBuilder(ctx, a) + if err != nil { + return "", fmt.Errorf("error creating ko builder: %w", err) + } + + koPublisher, err := b.newKoPublisher(ref) + if err != nil { + return "", fmt.Errorf("error creating ko publisher: %w", err) + } + defer koPublisher.Close() + + imageRef, err := b.buildAndPublish(ctx, a.ImageName, koBuilder, koPublisher) + if err != nil { + return "", fmt.Errorf("could not build and publish ko image %q: %w", a.ImageName, err) + } + fmt.Fprintln(out, imageRef.Name()) + + return b.getImageIdentifier(ctx, imageRef, ref) +} + +// buildAndPublish the image using the ko builder and publisher. +func (b *Builder) buildAndPublish(ctx context.Context, imageName string, koBuilder build.Interface, koPublisher publish.Interface) (name.Reference, error) { + importpath, err := getImportPath(imageName, koBuilder) + if err != nil { + return nil, fmt.Errorf("could not determine Go import path for ko image %q: %w", imageName, err) + } + imageMap, err := b.publishImages(ctx, []string{importpath}, koPublisher, koBuilder) + if err != nil { + return nil, fmt.Errorf("failed to publish ko image: %w", err) + } + imageRef, exists := imageMap[importpath] + if !exists { + return nil, fmt.Errorf("no built image found for Go import path %q build images: %+v", importpath, imageMap) + } + return imageRef, nil +} + +// getImportPath determines the Go import path that ko should build. +// +// If the image name from the Skaffold config has the prefix `ko://`, then +// treat the remainder of the string as the Go import path to build. This +// matches current ko behavior for working with Kubernetes resource files, and +// it will allow ko users to easily migrate to Skaffold without changing their +// Kubernetes YAML files. See https://github.com/google/ko#yaml-changes. +// +// If the image name does _not_ start with `ko://`, determine the Go import +// path of the image workspace directory. +func getImportPath(imageName string, koBuilder build.Interface) (string, error) { + if strings.HasPrefix(imageName, build.StrictScheme) { + return imageName, nil + } + return koBuilder.QualifyImport(".") +} + +// getImageIdentifier returns the image tag or digest for published images (`pushImages=true`), +// or the image ID from the local Docker daemon for sideloaded images (`pushImages=false`). +func (b *Builder) getImageIdentifier(ctx context.Context, imageRef name.Reference, ref string) (string, error) { + if b.pushImages { + return imageRef.Identifier(), nil + } + imageIdentifier, err := b.localDocker.ImageID(ctx, ref) + if err != nil { + return "", fmt.Errorf("could not get imageID from local Docker Daemon for image %s: %+v", ref, err) + } + return imageIdentifier, nil +} diff --git a/pkg/skaffold/build/ko/build_test.go b/pkg/skaffold/build/ko/build_test.go new file mode 100644 index 00000000000..80a09434aae --- /dev/null +++ b/pkg/skaffold/build/ko/build_test.go @@ -0,0 +1,153 @@ +/* +Copyright 2021 The Skaffold Authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package ko + +import ( + "bytes" + "context" + "strings" + "testing" + + "github.com/docker/docker/client" + "github.com/google/go-containerregistry/pkg/name" + "github.com/google/ko/pkg/build" + "github.com/google/ko/pkg/publish" + + // latestV1 "github.com/GoogleContainerTools/skaffold/pkg/skaffold/schema/latest/v1" + latestV1 "github.com/GoogleContainerTools/skaffold/pkg/skaffold/build/ko/schema" + "github.com/GoogleContainerTools/skaffold/pkg/skaffold/docker" + "github.com/GoogleContainerTools/skaffold/testutil" +) + +func TestBuildKoImages(t *testing.T) { + tests := []struct { + description string + ref string + imageID string + pushImages bool + importpath string + imageNameFromConfig string + workspace string + }{ + { + description: "simple image name and sideload image", + ref: "gcr.io/project-id/test-app1:testTag", + imageID: "imageID1", + pushImages: false, + importpath: "ko://github.com/GoogleContainerTools/skaffold/pkg/skaffold/build/ko", + imageNameFromConfig: "test-app1", + }, + { + description: "ko import path image name and sideload image", + ref: "gcr.io/project-id/example.com/myapp:myTag", + imageID: "imageID2", + pushImages: false, + importpath: "ko://example.com/myapp", + imageNameFromConfig: "ko://example.com/myapp", + }, + { + description: "simple image name and push image", + ref: "gcr.io/project-id/test-app2:testTag", + imageID: "testTag", + pushImages: true, + importpath: "ko://github.com/GoogleContainerTools/skaffold/pkg/skaffold/build/ko", + imageNameFromConfig: "test-app2", + }, + { + description: "ko import path image name and push image", + ref: "gcr.io/project-id/example.com/myapp:myTag", + imageID: "myTag", + pushImages: true, + importpath: "ko://example.com/myapp", + imageNameFromConfig: "ko://example.com/myapp", + }, + { + description: "workspace is not cwd", + ref: "gcr.io/project-id/example.com/test-app3:myTag", + imageID: "imageID3", + pushImages: false, + importpath: "ko://github.com/GoogleContainerTools/skaffold/pkg/skaffold/build/docker", + imageNameFromConfig: "test-app3", + workspace: "../docker", + }, + { + description: "ko import path image name and workspace is not cwd", + ref: "gcr.io/project-id/example.com/test-app4:myTag", + imageID: "imageID4", + pushImages: false, + importpath: "ko://github.com/GoogleContainerTools/skaffold/pkg/skaffold/build/docker", + imageNameFromConfig: "ko://github.com/GoogleContainerTools/skaffold/pkg/skaffold/build/docker", + workspace: "../docker", + }, + { + description: "ko import path image name and workspace is not cwd and import path is subdirectory of cwd", + ref: "gcr.io/project-id/example.com/test-app5:myTag", + imageID: "imageID5", + pushImages: false, + importpath: "ko://github.com/GoogleContainerTools/skaffold/pkg/skaffold/build/docker", + imageNameFromConfig: "ko://github.com/GoogleContainerTools/skaffold/pkg/skaffold/build/docker", + workspace: "..", + }, + } + for _, test := range tests { + testutil.Run(t, test.description, func(t *testutil.T) { + b := stubKoArtifactBuilder(test.ref, test.imageID, test.pushImages, test.importpath) + + artifact := &latestV1.Artifact{ + ArtifactType: latestV1.ArtifactType{ + KoArtifact: &latestV1.KoArtifact{}, + }, + Dependencies: []*latestV1.ArtifactDependency{}, + ImageName: test.imageNameFromConfig, + Workspace: test.workspace, + } + + var outBuffer bytes.Buffer + gotImageID, err := b.Build(context.TODO(), &outBuffer, artifact, test.ref) + t.CheckNoError(err) + if gotImageID != test.imageID { + t.Errorf("got image ID %s, wanted %s", gotImageID, test.imageID) + } + imageNameOut := strings.TrimSuffix(outBuffer.String(), "\n") + if imageNameOut != test.ref { + t.Errorf("image name output was %q, wanted %q", imageNameOut, test.ref) + } + }) + } +} + +func stubKoArtifactBuilder(ref string, imageID string, pushImages bool, importpath string) *Builder { + api := (&testutil.FakeAPIClient{}).Add(ref, imageID) + localDocker := fakeLocalDockerDaemon(api) + b := NewArtifactBuilder(localDocker, pushImages) + + // Fake implementation of the `publishImages` function. + b.publishImages = func(_ context.Context, _ []string, _ publish.Interface, _ build.Interface) (map[string]name.Reference, error) { + imageRef, err := name.ParseReference(ref) + if err != nil { + return nil, err + } + return map[string]name.Reference{ + importpath: imageRef, + }, nil + } + return b +} + +func fakeLocalDockerDaemon(api client.CommonAPIClient) docker.LocalDaemon { + return docker.NewLocalDaemon(api, nil, false, nil) +} diff --git a/pkg/skaffold/build/ko/builder.go b/pkg/skaffold/build/ko/builder.go new file mode 100644 index 00000000000..7bc3f78bf6a --- /dev/null +++ b/pkg/skaffold/build/ko/builder.go @@ -0,0 +1,45 @@ +/* +Copyright 2021 The Skaffold Authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package ko + +import ( + "context" + "strings" + + "github.com/google/ko/pkg/build" + "github.com/google/ko/pkg/commands" + "github.com/google/ko/pkg/commands/options" + + // latestV1 "github.com/GoogleContainerTools/skaffold/pkg/skaffold/schema/latest/v1" + latestV1 "github.com/GoogleContainerTools/skaffold/pkg/skaffold/build/ko/schema" + "github.com/GoogleContainerTools/skaffold/pkg/skaffold/version" +) + +func (b *Builder) newKoBuilder(ctx context.Context, a *latestV1.Artifact) (build.Interface, error) { + bo := buildOptions(a.KoArtifact.BaseImage, a.KoArtifact.Platforms, a.Workspace) + return commands.NewBuilder(ctx, bo) +} + +func buildOptions(baseImage string, platforms []string, workspace string) *options.BuildOptions { + return &options.BuildOptions{ + BaseImage: baseImage, + ConcurrentBuilds: 1, + Platform: strings.Join(platforms, ","), + UserAgent: version.UserAgentWithClient(), + WorkingDirectory: workspace, + } +} diff --git a/pkg/skaffold/build/ko/builder_test.go b/pkg/skaffold/build/ko/builder_test.go new file mode 100644 index 00000000000..0c9e986bfaa --- /dev/null +++ b/pkg/skaffold/build/ko/builder_test.go @@ -0,0 +1,75 @@ +/* +Copyright 2021 The Skaffold Authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package ko + +import ( + "testing" + + "github.com/GoogleContainerTools/skaffold/pkg/skaffold/version" + "github.com/GoogleContainerTools/skaffold/testutil" +) + +func TestBuildOptions(t *testing.T) { + tests := []struct { + description string + baseImage string + platforms []string + wantPlatform string + workspace string + }{ + { + description: "all zero value", + }, + { + description: "empty platforms", + platforms: []string{}, + }, + { + description: "base image", + baseImage: "gcr.io/distroless/static:nonroot", + }, + { + description: "multiple platforms", + platforms: []string{"linux/amd64", "linux/arm64"}, + wantPlatform: "linux/amd64,linux/arm64", + }, + { + description: "workspace", + workspace: "my-app-subdirectory", + }, + } + for _, test := range tests { + testutil.Run(t, test.description, func(t *testutil.T) { + bo := buildOptions(test.baseImage, test.platforms, test.workspace) + if bo.BaseImage != test.baseImage { + t.Errorf("wanted BaseImage (%q), got (%q)", test.baseImage, bo.BaseImage) + } + if bo.ConcurrentBuilds < 1 { + t.Errorf("ConcurrentBuilds must always be >= 1 for the ko builder") + } + if bo.Platform != test.wantPlatform { + t.Errorf("wanted platform (%q), got (%q)", test.wantPlatform, bo.Platform) + } + if bo.UserAgent != version.UserAgentWithClient() { + t.Errorf("need user agent for fetching the base image") + } + if bo.WorkingDirectory != test.workspace { + t.Errorf("wanted WorkingDirectory (%q), got (%q)", test.workspace, bo.WorkingDirectory) + } + }) + } +} diff --git a/pkg/skaffold/build/ko/ko.go b/pkg/skaffold/build/ko/ko.go index 4fa7e5fb5f3..e53db9fd410 100644 --- a/pkg/skaffold/build/ko/ko.go +++ b/pkg/skaffold/build/ko/ko.go @@ -17,9 +17,30 @@ limitations under the License. package ko import ( - kobuild "github.com/google/ko/pkg/build" + "context" + + "github.com/google/go-containerregistry/pkg/name" + "github.com/google/ko/pkg/build" + "github.com/google/ko/pkg/commands" + "github.com/google/ko/pkg/publish" + + "github.com/GoogleContainerTools/skaffold/pkg/skaffold/docker" ) -// KoScheme is the prefix used to disambiguate image references and Go import paths. -// Adding the const here to force import of a ko package. -const KoScheme = kobuild.StrictScheme +// Builder is an artifact builder that uses ko +type Builder struct { + localDocker docker.LocalDaemon + pushImages bool + + // publishImages can be overridden for unit testing purposes. + publishImages func(context.Context, []string, publish.Interface, build.Interface) (map[string]name.Reference, error) +} + +// NewArtifactBuilder returns a new ko artifact builder +func NewArtifactBuilder(localDocker docker.LocalDaemon, pushImages bool) *Builder { + return &Builder{ + localDocker: localDocker, + pushImages: pushImages, + publishImages: commands.PublishImages, + } +} diff --git a/pkg/skaffold/build/ko/ko_test.go b/pkg/skaffold/build/ko/ko_test.go index 5f89d51bf2b..566a1a97bac 100644 --- a/pkg/skaffold/build/ko/ko_test.go +++ b/pkg/skaffold/build/ko/ko_test.go @@ -18,12 +18,11 @@ package ko import ( "testing" - - kobuild "github.com/google/ko/pkg/build" ) -func TestStub(t *testing.T) { - if KoScheme != kobuild.StrictScheme { - t.Fatal() +func TestNewArtifactBuilderCanPublishImages(t *testing.T) { + b := NewArtifactBuilder(nil, true) + if b.publishImages == nil { + t.Errorf("constructor function should populate publishImages func") } } diff --git a/pkg/skaffold/build/ko/publisher.go b/pkg/skaffold/build/ko/publisher.go new file mode 100644 index 00000000000..5e4e03fd172 --- /dev/null +++ b/pkg/skaffold/build/ko/publisher.go @@ -0,0 +1,53 @@ +/* +Copyright 2021 The Skaffold Authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package ko + +import ( + "github.com/google/go-containerregistry/pkg/name" + "github.com/google/go-containerregistry/pkg/v1/daemon" + "github.com/google/ko/pkg/commands" + "github.com/google/ko/pkg/commands/options" + "github.com/google/ko/pkg/publish" + + "github.com/GoogleContainerTools/skaffold/pkg/skaffold/version" +) + +func (b *Builder) newKoPublisher(ref string) (publish.Interface, error) { + po, err := publishOptions(ref, b.pushImages, b.localDocker.RawClient()) + if err != nil { + return nil, err + } + return commands.NewPublisher(po) +} + +func publishOptions(ref string, pushImages bool, dockerClient daemon.Client) (*options.PublishOptions, error) { + imageRef, err := name.ParseReference(ref) + if err != nil { + return nil, err + } + imageNameWithoutTag := imageRef.Context().Name() + return &options.PublishOptions{ + Bare: true, + DockerClient: dockerClient, + DockerRepo: imageNameWithoutTag, + Local: !pushImages, + LocalDomain: imageNameWithoutTag, + Push: pushImages, + Tags: []string{imageRef.Identifier()}, + UserAgent: version.UserAgentWithClient(), + }, nil +} diff --git a/pkg/skaffold/build/ko/publisher_test.go b/pkg/skaffold/build/ko/publisher_test.go new file mode 100644 index 00000000000..dc537378b27 --- /dev/null +++ b/pkg/skaffold/build/ko/publisher_test.go @@ -0,0 +1,84 @@ +/* +Copyright 2021 The Skaffold Authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package ko + +import ( + "testing" + + "github.com/GoogleContainerTools/skaffold/pkg/skaffold/version" + "github.com/GoogleContainerTools/skaffold/testutil" +) + +func TestPublishOptions(t *testing.T) { + tests := []struct { + description string + ref string + pushImages bool + repo string + tag string + }{ + { + description: "sideloaded image", + ref: "registry.example.com/repository/myapp1:tag1", + pushImages: false, + repo: "registry.example.com/repository/myapp1", + tag: "tag", + }, + { + description: "published image", + ref: "registry.example.com/repository/myapp2:tag2", + pushImages: true, + repo: "registry.example.com/repository/myapp2", + tag: "tag", + }, + } + for _, test := range tests { + testutil.Run(t, test.description, func(t *testutil.T) { + dockerClient := fakeDockerAPIClient(test.ref, "imageID") + po, err := publishOptions(test.ref, test.pushImages, dockerClient) + t.CheckNoError(err) + if !po.Bare || po.BaseImportPaths || po.PreserveImportPaths { + t.Errorf("use ko's Bare naming option as it allow for arbitrary image names") + } + if po.DockerClient != dockerClient { + t.Errorf("use provided docker client") + } + if test.pushImages && po.DockerRepo != test.repo { + t.Errorf("wanted DockerRepo (%q), got (%q)", test.repo, po.DockerRepo) + } + if !test.pushImages && po.LocalDomain != test.repo { + t.Errorf("wanted LocalDomain (%q), got (%q)", test.repo, po.DockerRepo) + } + if test.pushImages == po.Local { + t.Errorf("Local (%v) should be the inverse of pushImages (%v)", po.Local, test.pushImages) + } + if test.pushImages != po.Push { + t.Errorf("Push (%v) should match pushImages (%v)", po.Push, test.pushImages) + } + if len(po.Tags) != 1 && po.Tags[0] != test.tag { + t.Errorf("wanted Tags (%+v), got (%+v)", []string{test.tag}, po.Tags) + } + if po.UserAgent != version.UserAgentWithClient() { + t.Errorf("wanted UserAgent (%s), got (%s)", version.UserAgentWithClient(), po.UserAgent) + } + }) + } +} + +func fakeDockerAPIClient(ref string, imageID string) *testutil.FakeAPIClient { + return (&testutil.FakeAPIClient{}).Add(ref, imageID) +} diff --git a/pkg/skaffold/build/ko/schema/temporary.go b/pkg/skaffold/build/ko/schema/temporary.go new file mode 100644 index 00000000000..e9c553709d1 --- /dev/null +++ b/pkg/skaffold/build/ko/schema/temporary.go @@ -0,0 +1,84 @@ +/* +Copyright 2021 The Skaffold Authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Temporary home for schema changes. + +package schema + +// KoArtifact builds images using [ko](https://github.com/google/ko). +type KoArtifact struct { + // BaseImage overrides the default ko base image (`gcr.io/distroless/static:nonroot`). + // Corresponds to, and overrides, the `defaultBaseImage` in `.ko.yaml`. + BaseImage string `yaml:"fromImage,omitempty"` + + // Dependencies are the file dependencies that Skaffold should watch for both rebuilding and file syncing for this artifact. + Dependencies *KoDependencies `yaml:"dependencies,omitempty"` + + // Labels are key-value string pairs to add to the image config. + // For example: `{"foo":"bar"}`. + Labels map[string]string `yaml:"labels,omitempty"` + + // Platforms is the list of platforms to build images for. Each platform + // is of the format `os[/arch[/variant]]`, e.g., `linux/amd64`. + // Defaults to `all` to build for all platforms supported by the + // base image. + Platforms []string `yaml:"platforms,omitempty"` +} + +// KoDependencies is used to specify dependencies for an artifact built by ko. +type KoDependencies struct { + // Paths should be set to the file dependencies for this artifact, so that the skaffold file watcher knows when to rebuild and perform file synchronization. + // Defaults to {"go.mod", "**.go"}. + Paths []string `yaml:"paths,omitempty" yamltags:"oneOf=dependency"` + + // Ignore specifies the paths that should be ignored by skaffold's file watcher. + // If a file exists in both `paths` and in `ignore`, it will be ignored, and will be excluded from both rebuilds and file synchronization. + Ignore []string `yaml:"ignore,omitempty"` +} + +// Artifact are the items that need to be built, along with the context in which +// they should be built. +type Artifact struct { + // ImageName is the name of the image to be built. + // For example: `gcr.io/k8s-skaffold/example`. + ImageName string `yaml:"image,omitempty" yamltags:"required"` + + // Workspace is the directory containing the artifact's sources. + // Defaults to `.`. + Workspace string `yaml:"context,omitempty" skaffold:"filepath"` + + // ArtifactType describes how to build an artifact. + ArtifactType `yaml:",inline"` + + // Dependencies describes build artifacts that this artifact depends on. + Dependencies []*ArtifactDependency `yaml:"requires,omitempty"` +} + +// ArtifactType describes how to build an artifact. +type ArtifactType struct { + // KoArtifact builds images using [ko](https://github.com/google/ko). + KoArtifact *KoArtifact `yaml:"-,omitempty" yamltags:"oneOf=artifact"` +} + +// ArtifactDependency describes a specific build dependency for an artifact. +type ArtifactDependency struct { + // ImageName is a reference to an artifact's image name. + ImageName string `yaml:"image" yamltags:"required"` + // Alias is a token that is replaced with the image reference in the builder definition files. + // For example, the `docker` builder will use the alias as a build-arg key. + // Defaults to the value of `image`. + Alias string `yaml:"alias,omitempty"` +}