imagetools inspect: add --format flag #854

crazy-max · 2021-11-20T09:18:50Z

adds --format flag to imagetools inspect command. Currently supported values are manifest (default and current behavior) and buildinfo.

buildinfo is the new one to retrieve build information for an image in the registry.

$ docker buildx imagetools inspect --format buildinfo crazymax/buildkit-buildsources-test:buildx
Name:    docker.io/crazymax/buildkit-buildsources-test:buildx

Sources:
  Type: docker-image
  Ref:  docker.io/docker/buildx-bin:0.6.1@sha256:a652ced4a4141977c7daaed0a074dcd9844a78d7d2615465b12f433ae6dd29f0
  Pin:  sha256:a652ced4a4141977c7daaed0a074dcd9844a78d7d2615465b12f433ae6dd29f0

  Type: docker-image
  Ref:  docker.io/library/alpine:3.13
  Pin:  sha256:026f721af4cf2843e07bba648e158fb35ecc876d822130633cc49f707f0fc88c

  Type: docker-image
  Ref:  docker.io/moby/buildkit:v0.9.0
  Pin:  sha256:8dc668e7f66db1c044aadbed306020743516a94848793e0f81f94a087ee78cab

  Type: docker-image
  Ref:  docker.io/tonistiigi/xx@sha256:21a61be4744f6531cb5f33b0e6f40ede41fa3a1b8c82d5946178f80cc84bfc04
  Pin:  sha256:21a61be4744f6531cb5f33b0e6f40ede41fa3a1b8c82d5946178f80cc84bfc04

  Type: git
  Ref:  https://github.com/crazy-max/buildkit-buildsources-test.git#master
  Pin:  c7f9447d38471b486e248857a89143776d27096c

  Type: http
  Ref:  https://raw.githubusercontent.com/moby/moby/master/README.md
  Pin:  sha256:419455202b0ef97e480d7f8199b26a721a417818bc0e2d106975f74323f25e6c

cc @JamieMagee

Signed-off-by: CrazyMax [email protected]

crazy-max · 2021-11-20T11:20:17Z

util/imagetools/inspect.go

+	// needs to discard containerd logger otherwise it will print
+	// unnecessary info during image reference resolution like
+	// https://github.com/containerd/containerd/blob/1a88cf5242445657258e0c744def5017d7cfb492/remotes/docker/resolver.go#L288
+	ctx = bxctx.WithLoggerDiscarded(ctx)


needs to discard containerd logger otherwise it will print unnecessary info during image reference resolution:

$ docker buildx imagetools inspect alpinefoo INFO[0001] trying next host error="pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed" host=registry-1.docker.io error: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

now:

$ docker buildx imagetools inspect alpinefoo error: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

crazy-max · 2021-11-20T12:34:30Z

In addition, wondering if we could add another flag to display buildinfo from a metadata file (--metadata-file) generated by the build command.

JamieMagee · 2021-11-22T18:15:59Z

@crazy-max Thanks for this!

What's the typical release cadence for buildx? And when could I expect to see this functionality in a docker cli or docker ce release?

crazy-max · 2021-11-23T12:36:42Z

@tonistiigi As discussed, --format flag has been added to the imagetools inspect command to handle build information. In a follow-up we could add a config format to display the image configuration.

@JamieMagee

What's the typical release cadence for buildx? And when could I expect to see this functionality in a docker cli or docker ce release?

The buildinfo feature is only available in the main branch of the BuildKit repo atm. Not sure when the first 0.10 RC of BuildKit will be released. I think @tonistiigi and @sipsma want to consolidate the MergeOp feature and provide COPY with merge semantics.

About buildx, there will be a patch release on wednesday (0.7.1) to fix the current issues. This PR will probably be available in the next minor release but you can already download the artifacts in staging from the GitHub workflow if you want to try it.

tonistiigi · 2021-11-23T18:30:19Z

I'd expect the template format like other commands. Eg. {{json .Config}}, {{json .BuildInfo}}

AkihiroSuda · 2021-11-24T02:04:52Z

commands/imagetools/inspect.go

@@ -85,7 +134,8 @@ func inspectCmd(dockerCli command.Cli, rootOpts RootOptions) *cobra.Command {
 	}

 	flags := cmd.Flags()
-	flags.BoolVar(&options.raw, "raw", false, "Show original JSON manifest")
+	flags.StringVar(&options.format, "format", "manifest", "Format the output")


Suggested change

flags.StringVar(&options.format, "format", "manifest", "Format the output")

flags.StringVar(&options.format, "format", "manifest", "Format the output (\"manifest\", or \"buildinfo\")")

crazy-max · 2021-11-24T08:02:05Z

I'd expect the template format like other commands. Eg. {{json .Config}}, {{json .BuildInfo}}

Agree but then --raw should be removed if we want to be strict and follow the current behavior of docker/cli like image inspect.

tonistiigi · 2021-11-24T08:25:58Z

--raw is not the same as {{json .}}. It is the data that can be used to validate the checksum. We could move it to another field but it does have a very specific use case.

crazy-max · 2021-11-24T08:36:34Z

Ok so {{json .}} would print manifest + config + buildinfo right? Currently we display the manifest in raw format so I would guess the default value for format would be {{raw .Manifest}} for backward compat. I guess we should implement a formatter like the one from #830. WDYT?

tonistiigi · 2021-12-06T06:22:48Z

{{json .}} printing all in json format, {{.buildinfo}} printing the buildinfo block. {{json .config}} printing the config as json would seem logical to me. {{.manifest}} I guess is the same but a bit weird that it is either index or manifest.

For --raw I think we can make it just incompatible with --format. Theoretically --raw --format {{json .config}}` could print the raw value of config, but that seems too special case to be worth of a custom implementation.

Is the config taking the first item from the index atm?

crazy-max · 2021-12-17T15:34:40Z

@tonistiigi

Sorry didn't see your comment!

{{json .}} printing all in json format, {{.buildinfo}} printing the buildinfo block. {{json .config}} printing the config as json would seem logical to me. {{.manifest}} I guess is the same but a bit weird that it is either index or manifest.

For --raw I think we can make it just incompatible with --format. Theoretically --raw --format {{json .config}}` could print the raw value of config, but that seems too special case to be worth of a custom implementation.

Ok sgtm

Is the config taking the first item from the index atm?

Atm I don't handle config, only buildinfo and manifest.

tonistiigi

» docker buildx imagetools inspect --raw alpine
error: format and raw cannot be used together

tonistiigi

For multi-platform this breaks:

docker buildx imagetools inspect  mysql
error: no match for platform in manifest sha256:0962b771c2398c6dcddbbe77b3cf6658408396229b612035d938fb7c8d11c23c: not found

Should also make sure that without format we don't make these requests at all.

util/imagetools/printers.go

tonistiigi · 2022-02-25T07:34:00Z

As a follow-up, looks like some pkg reorganization is needed as these functions definitely should not require importing ttrpc etc.

Signed-off-by: CrazyMax <[email protected]>

tonistiigi · 2022-02-25T19:10:18Z

docs/reference/buildx_imagetools_inspect.md

+
+* `.Name`: provides the reference of the image
+* `.Manifest`: provides manifest or manifest list
+* `.Image`: provides the image config as `application/vnd.oci.image.config.v1+json` mediatype


what does this "as mediatype" mean? I expect it is just the full config as it is in registry?

Yes that's it I will remove that. Was taken from https://github.com/opencontainers/image-spec/blob/1f308fd27264a52f6a089202b1ff9dc15ff27398/specs-go/v1/config.go#L81

Signed-off-by: CrazyMax <[email protected]>

crazy-max marked this pull request as ready for review November 20, 2021 09:28

crazy-max requested a review from tonistiigi November 20, 2021 09:28

crazy-max force-pushed the buildinfo-cmd branch from 814ee07 to d5a04b3 Compare November 20, 2021 11:17

crazy-max commented Nov 20, 2021

View reviewed changes

crazy-max force-pushed the buildinfo-cmd branch 2 times, most recently from a210432 to 09894eb Compare November 20, 2021 12:22

crazy-max changed the title ~~imagetools: buildinfo cmd~~ imagetools inspect: add --format flag Nov 23, 2021

crazy-max force-pushed the buildinfo-cmd branch 3 times, most recently from 1330560 to 90f7d4b Compare November 23, 2021 12:05

AkihiroSuda reviewed Nov 24, 2021

View reviewed changes

crazy-max force-pushed the buildinfo-cmd branch from 90f7d4b to 0537928 Compare December 1, 2021 21:09

crazy-max marked this pull request as draft December 9, 2021 11:43

crazy-max force-pushed the buildinfo-cmd branch 6 times, most recently from b4b65de to b1019c7 Compare February 24, 2022 15:33

crazy-max marked this pull request as ready for review February 24, 2022 15:40

crazy-max force-pushed the buildinfo-cmd branch from b1019c7 to 0a28e6d Compare February 24, 2022 16:28

tonistiigi reviewed Feb 25, 2022

View reviewed changes

util/imagetools/printers.go Outdated Show resolved Hide resolved

crazy-max force-pushed the buildinfo-cmd branch from cbc8508 to addb396 Compare February 25, 2022 11:40

crazy-max requested a review from tonistiigi February 25, 2022 11:40

imagetools inspect: add --format flag

6c56109

Signed-off-by: CrazyMax <[email protected]>

crazy-max force-pushed the buildinfo-cmd branch from addb396 to f8ae387 Compare February 25, 2022 16:59

tonistiigi reviewed Feb 25, 2022

View reviewed changes

crazy-max force-pushed the buildinfo-cmd branch from f8ae387 to 5f807b9 Compare February 25, 2022 19:21

tonistiigi approved these changes Feb 25, 2022

View reviewed changes

imagetools inspect: multi-platform support

20a55e9

Signed-off-by: CrazyMax <[email protected]>

crazy-max force-pushed the buildinfo-cmd branch from 5f807b9 to 20a55e9 Compare February 25, 2022 19:30

crazy-max added this to the v0.8.0 milestone Feb 25, 2022

tonistiigi merged commit 7af2980 into docker:master Feb 25, 2022

crazy-max deleted the buildinfo-cmd branch February 26, 2022 02:58

crazy-max mentioned this pull request Feb 26, 2022

imagetools inspect: keep platform order #972

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

imagetools inspect: add --format flag #854

imagetools inspect: add --format flag #854

crazy-max commented Nov 20, 2021 •

edited

Loading

crazy-max Nov 20, 2021

crazy-max commented Nov 20, 2021 •

edited

Loading

JamieMagee commented Nov 22, 2021

crazy-max commented Nov 23, 2021 •

edited

Loading

tonistiigi commented Nov 23, 2021

AkihiroSuda Nov 24, 2021

crazy-max commented Nov 24, 2021 •

edited

Loading

tonistiigi commented Nov 24, 2021

crazy-max commented Nov 24, 2021 •

edited

Loading

tonistiigi commented Dec 6, 2021

crazy-max commented Dec 17, 2021

tonistiigi left a comment

tonistiigi left a comment •

edited

Loading

tonistiigi commented Feb 25, 2022

tonistiigi Feb 25, 2022 •

edited

Loading

crazy-max Feb 25, 2022

	flags.StringVar(&options.format, "format", "manifest", "Format the output")
	flags.StringVar(&options.format, "format", "manifest", "Format the output (\"manifest\", or \"buildinfo\")")

imagetools inspect: add --format flag #854

imagetools inspect: add --format flag #854

Conversation

crazy-max commented Nov 20, 2021 • edited Loading

crazy-max Nov 20, 2021

Choose a reason for hiding this comment

crazy-max commented Nov 20, 2021 • edited Loading

JamieMagee commented Nov 22, 2021

crazy-max commented Nov 23, 2021 • edited Loading

tonistiigi commented Nov 23, 2021

AkihiroSuda Nov 24, 2021

Choose a reason for hiding this comment

crazy-max commented Nov 24, 2021 • edited Loading

tonistiigi commented Nov 24, 2021

crazy-max commented Nov 24, 2021 • edited Loading

tonistiigi commented Dec 6, 2021

crazy-max commented Dec 17, 2021

tonistiigi left a comment

Choose a reason for hiding this comment

tonistiigi left a comment • edited Loading

Choose a reason for hiding this comment

tonistiigi commented Feb 25, 2022

tonistiigi Feb 25, 2022 • edited Loading

Choose a reason for hiding this comment

crazy-max Feb 25, 2022

Choose a reason for hiding this comment

crazy-max commented Nov 20, 2021 •

edited

Loading

crazy-max commented Nov 20, 2021 •

edited

Loading

crazy-max commented Nov 23, 2021 •

edited

Loading

crazy-max commented Nov 24, 2021 •

edited

Loading

crazy-max commented Nov 24, 2021 •

edited

Loading

tonistiigi left a comment •

edited

Loading

tonistiigi Feb 25, 2022 •

edited

Loading