(DRON-232) enable build-kit for secrets consumption

[tphoney]

Using secrets from a file with buildkit

docker run --rm \
  -e PLUGIN_TAG=latest \
  -e PLUGIN_REPO=octocat/hello-world \
  -e PLUGIN_SECRET=id=mysecret,src=secret-file \
  -e DRONE_COMMIT_SHA=d8dbe4d94f15fe89232e0402c6e8a0ddf21af3ab \
  -v $(pwd):$(pwd) \
  -w $(pwd) \
  --privileged \
  plugins/docker --dry-run
+ /usr/local/bin/dockerd --data-root /var/lib/docker --host=unix:///var/run/docker.sock
Registry credentials or Docker config not provided. Guest mode enabled.
+ /usr/local/bin/docker version
Client:
 Version:           20.10.9
 API version:       1.41
 Go version:        go1.16.8
 Git commit:        c2ea9bc
 Built:             Mon Oct  4 16:03:22 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.9
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.8
  Git commit:       79ea9d3
  Built:            Mon Oct  4 16:07:30 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.4.11
  GitCommit:        5b46e404f6b9f661a205e28d59c982d3634148f8
 runc:
  Version:          1.0.2
  GitCommit:        v1.0.2-0-g52b36a2d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
+ /usr/local/bin/docker info
Client:
 Context:    default
 Debug Mode: false

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 0
 Server Version: 20.10.9
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 5b46e404f6b9f661a205e28d59c982d3634148f8
 runc version: v1.0.2-0-g52b36a2d
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.10.93.2-microsoft-standard-WSL2
 Operating System: Alpine Linux v3.14 (containerized)
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 19.54GiB
 Name: b0e4a138cb48
 ID: 5FLC:OYBN:HZDL:4MYA:UJFF:XOM2:UU2H:HLTG:HWOU:JCVQ:AXJQ:4GQ7
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support
+ /usr/local/bin/docker build --rm=true -f Dockerfile -t d8dbe4d94f15fe89232e0402c6e8a0ddf21af3ab . --pull=true --secret id=mysecret,src=secret-file --label org.opencontainers.image.created=2022-02-11T10:46:47Z --label org.opencontainers.image.revision=d8dbe4d94f15fe89232e0402c6e8a0ddf21af3ab --label org.opencontainers.image.source= --label org.opencontainers.image.url=
#1 [internal] load build definition from Dockerfile
#1 sha256:ac4f806e480bfc6c03a8cbb38abe11817990f5d99dcb5338e8d8b12f034a1f94
#1 transferring dockerfile: 197B done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 sha256:7db37e9a0017c812a33c344ec6a638f6b503d8f75d733f6d0729bc1c4df3ce81
#2 transferring context: 2B done
#2 DONE 0.0s

#3 resolve image config for docker.io/docker/dockerfile:1.2
#3 sha256:b239a20f31d7f1e5744984df3d652780f1a82c37554dd73e1ad47c8eb05b0d69
#3 DONE 3.7s

#4 docker-image://docker.io/docker/dockerfile:1.2@sha256:e2a8561e419ab1ba6b2fe6cbdf49fd92b95912df1cf7d313c3e2230a333fdbcc
#4 sha256:37e0c519b0431ef5446f4dd0a4588ba695f961e9b0e800cd8c7f5ba6165af727
#4 resolve docker.io/docker/dockerfile:1.2@sha256:e2a8561e419ab1ba6b2fe6cbdf49fd92b95912df1cf7d313c3e2230a333fdbcc done
#4 sha256:3cc8e449ce9f6e0752ede8f50a7334bf0c7b2d24d76da2ffae7aa6a729dd1da4 0B / 9.64MB 0.1s
#4 sha256:e2a8561e419ab1ba6b2fe6cbdf49fd92b95912df1cf7d313c3e2230a333fdbcc 1.69kB / 1.69kB done
#4 sha256:e3ee2e6b536452d876b1c5aa12db9bca51b8f52b2505178cae6d13e33daeed2b 528B / 528B done
#4 sha256:86e43bba076d67c1a890cbc07813806b11eca53843dc643202d939b986c8c332 1.21kB / 1.21kB done
#4 sha256:3cc8e449ce9f6e0752ede8f50a7334bf0c7b2d24d76da2ffae7aa6a729dd1da4 1.05MB / 9.64MB 0.5s
#4 sha256:3cc8e449ce9f6e0752ede8f50a7334bf0c7b2d24d76da2ffae7aa6a729dd1da4 2.10MB / 9.64MB 0.7s
#4 sha256:3cc8e449ce9f6e0752ede8f50a7334bf0c7b2d24d76da2ffae7aa6a729dd1da4 3.15MB / 9.64MB 1.1s
#4 sha256:3cc8e449ce9f6e0752ede8f50a7334bf0c7b2d24d76da2ffae7aa6a729dd1da4 4.19MB / 9.64MB 1.4s
#4 sha256:3cc8e449ce9f6e0752ede8f50a7334bf0c7b2d24d76da2ffae7aa6a729dd1da4 5.24MB / 9.64MB 1.9s
#4 sha256:3cc8e449ce9f6e0752ede8f50a7334bf0c7b2d24d76da2ffae7aa6a729dd1da4 6.29MB / 9.64MB 2.2s
#4 sha256:3cc8e449ce9f6e0752ede8f50a7334bf0c7b2d24d76da2ffae7aa6a729dd1da4 7.34MB / 9.64MB 2.6s

  1 (DRON-232) enable build-kit for secrets consumption
#4 sha256:3cc8e449ce9f6e0752ede8f50a7334bf0c7b2d24d76da2ffae7aa6a729dd1da4 8.39MB / 9.64MB 3.0s
#4 sha256:3cc8e449ce9f6e0752ede8f50a7334bf0c7b2d24d76da2ffae7aa6a729dd1da4 9.44MB / 9.64MB 3.2s
#4 sha256:3cc8e449ce9f6e0752ede8f50a7334bf0c7b2d24d76da2ffae7aa6a729dd1da4 9.64MB / 9.64MB 3.3s done
#4 extracting sha256:3cc8e449ce9f6e0752ede8f50a7334bf0c7b2d24d76da2ffae7aa6a729dd1da4 0.1s
#4 extracting sha256:3cc8e449ce9f6e0752ede8f50a7334bf0c7b2d24d76da2ffae7aa6a729dd1da4 0.1s done
#4 DONE 3.5s

#5 [internal] load metadata for docker.io/library/alpine:latest
#5 sha256:d4fb25f5b5c00defc20ce26f2efc4e288de8834ed5aa59dff877b495ba88fda6
#5 DONE 3.7s

#7 [1/2] FROM docker.io/library/alpine@sha256:21a3deaa0d32a8057914f36584b5288d2e5ecc984380bc0118285c70fa8c9300
#7 sha256:97bad1e2d771db4bb4f0a28b234a1af153c7d6fe5b02e7669d9a7da2c2e18a63
#7 resolve docker.io/library/alpine@sha256:21a3deaa0d32a8057914f36584b5288d2e5ecc984380bc0118285c70fa8c9300 done
#7 sha256:21a3deaa0d32a8057914f36584b5288d2e5ecc984380bc0118285c70fa8c9300 1.64kB / 1.64kB done
#7 sha256:e7d88de73db3d3fd9b2d63aa7f447a10fd0220b7cbf39803c803f2af9ba256b3 528B / 528B done
#7 sha256:c059bfaa849c4d8e4aecaeb3a10c2d9b3d85f5165c66ad3a4d937758128c4d18 1.47kB / 1.47kB done
#7 sha256:59bf1c3509f33515622619af21ed55bbe26d24913cedbca106468a5fb37a50c3 0B / 2.82MB 0.1s
#7 sha256:59bf1c3509f33515622619af21ed55bbe26d24913cedbca106468a5fb37a50c3 1.05MB / 2.82MB 0.4s
#7 sha256:59bf1c3509f33515622619af21ed55bbe26d24913cedbca106468a5fb37a50c3 2.10MB / 2.82MB 0.7s
#7 extracting sha256:59bf1c3509f33515622619af21ed55bbe26d24913cedbca106468a5fb37a50c3 0.1s done
#7 sha256:59bf1c3509f33515622619af21ed55bbe26d24913cedbca106468a5fb37a50c3 2.82MB / 2.82MB 0.9s done
#7 DONE 1.1s

#6 [2/2] RUN --mount=type=secret,id=mysecret cat /run/secrets/mysecret
#6 sha256:8c26d2b932e7c35ee5c8e9a380c2e740af2763f2cf7927608318f8779a469ffe
#6 0.504 COOL BANANAS
#6 DONE 0.5s

#8 exporting to image
#8 sha256:e8c613e07b0b7ff33893b694f7759a10d42e180f2b4dc349fb57dc6b71dcab00
#8 exporting layers 0.0s done
#8 writing image sha256:6d18a5587de1f83b29875e1241964a5f947a84de08281a34f5672000ee61d4ef done
#8 naming to docker.io/library/d8dbe4d94f15fe89232e0402c6e8a0ddf21af3ab done
#8 DONE 0.0s
+ /usr/local/bin/docker tag d8dbe4d94f15fe89232e0402c6e8a0ddf21af3ab octocat/hello-world:latest
+ /usr/local/bin/docker rmi d8dbe4d94f15fe89232e0402c6e8a0ddf21af3ab
+ /usr/local/bin/docker system prune -f

[eoinmcafee00]

LGTM

[ste93cry]

I was looking at the changes as I'm interested in the feature and since the new setting is a string, there isn't a way to set multiple secrets. Is there any plan to support this use case?

[tphoney]

Hey @ste93cry we were trying this out for a customer, and haven't really fully documented it as yet.
Basically it follows on from dockers cli https://docs.docker.com/develop/develop-images/build_enhancements/#new-docker-build-secret-information
We hadnt planned on adding that straight away. PR's welcome :)
I think changing

drone-docker/docker.go

Line 61 in d0b9da3

Secret string // secret keypair

to an array
And

drone-docker/docker.go

Lines 306 to 308 in d0b9da3

	if build.Secret != "" {
	args = append(args, "--secret", build.Secret)
	}

to a for loop would do the trick.
Any help is welcome.

[ste93cry]

Yup, that was exactly what I meant. I will try to contribute with a PR ASAP. In the meanwhile, do you have any ETA on when this feature is scheduled for release?

[tphoney]

So the current secret feature is on the latest tag, we were letting it soak, to see if there were any issues.
If you make the PR to handle multiple build secrets, i will do a release with it in.
Sound like deal ?

[ste93cry]

Yup, that would be great!