mquery images contain wrong architecture in image config

[tonistiigi]

 # docker buildx imagetools inspect mplatform/mquery
Name:      docker.io/mplatform/mquery:latest
MediaType: application/vnd.docker.distribution.manifest.list.v2+json
Digest:    sha256:e15189e3d6fbcee8a6ad2ef04c1ec80420ab0fdcf0d70408c0e914af80dfb107

Manifests:
  Name:      docker.io/mplatform/mquery:latest@sha256:84cf13c9dc46373fdd0ab1bf5459da0b644d8367c6e7149c22979fc366190631
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/amd64

  Name:      docker.io/mplatform/mquery:latest@sha256:f55b14d9b031d38637d8d277364902eba603f76b7a9c42adc02a33d5d86799a1
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/arm

  Name:      docker.io/mplatform/mquery:latest@sha256:7fee21210e3505df0a85bb19cd41a51fe1be5818d5d45f4ca42af232acf8b030
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/arm64

...

Pick arm64

 # docker pull docker.io/mplatform/mquery:latest@sha256:7fee21210e3505df0a85bb19cd41a51fe1be5818d5d45f4ca42af232acf8b030
...
 # docker inspect -f '{{json .Architecture}}' docker.io/mplatform/mquery:latest@sha256:7fee21210e3505df0a85bb19cd41a51fe1be5818d5d45f4ca42af232acf8b030
"amd64

Same for linux/arm image.

In Docker 20.10 this means users get the following warning when running:

WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

[StefanScherer]

It looked weird on Apple Silicon M1 machines, and even weirder on a Raspberry Pi when I saw this warning that linux/amd64 is the requested platform. The image actually can run😅

And I found the problem in one of my multi-arch+multi-os images. My CI pipeline creates the manifest list with docker manifest annotate and docker manifest push commands (https://github.com/StefanScherer/whoami/blob/main/deploy.ps1#L93-L95)
I even annotate the arm64 image with v8. 🤔

@tonistiigi I fear that buildx won't help for multi-os images with one or more Windows images in the manifest list. Any quick tip how we can fix the architecture when we build the arm64 image on an amd64 CI machines?

[tonistiigi]

Any quick tip how we can fix the architecture when we build the arm64 image on an amd64 CI machines?

As long as buildkit is used with correct --platform and no architecture was hardcoded in Dockerfile this should not happen. This image looks to be built 3 years ago with some other method.

I fear that buildx won't help for multi-os images with one or more Windows images in the manifest list

Buildx with container/kubernetes driver only builds windows images for cross compilation cases (eg. copying files from linux stage to a windows stage image). It can't run wcow containers. There is docker buildx imagetools create for the case when you want to merge already built images(as an alternative to manifest annotate or Phil's manifest-tool). But again, this is not what caused the current issue as neither of these tools modifies the image config afaik.

[estesp]

So, I think the root of the issue is that these images are all built on amd64 as cross-built Go binaries which then are assembled in a manifest list and pushed to mplatform/mquery: https://github.com/estesp/mquery/blob/master/.travis.yml#L16-L21

[StefanScherer]

Thanks, I‘ll look into buildx imagetools command.
The arm64 image is classically built withdocker build on Travis with amd64 CPU. https://github.com/estesp/mquery/blob/master/.travis.yml#L19 and then the manifest-tool

AFAIK there was no way to fix the architecture, so buildx is the way to go.

[estesp]

Fixed in v0.4.0, which has been pushed as the latest tag to the mplatform/mquery image. Thanks to buildkit for the feature to be able to finally build these properly :)