multi stage image with --platform - final image is of build architecture

[grooverdan]

Interestingly docker --platform takes the first FROM as the target architecture and others at the native architecture:

Dockerfile.platformargs:

FROM alpine
ARG BUILDPLATFORM
ARG BUILDOS
ARG BUILDARCH
ARG BUILDVARIANT
ARG TARGETPLATFORM
ARG TARGETOS
ARG TARGETARCH
ARG TARGETVARIANT
FROM ubuntu
COPY --from=alpine /bin/ls /tmp/ls

$ DOCKER_BUILDKIT=1  docker version
Client:
 Version:           18.09.5
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        e8ff056
 Built:             Thu Apr 11 04:44:27 2019
 OS/Arch:           linux/amd64
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          18.09.5
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.8
  Git commit:       e8ff056
  Built:            Thu Apr 11 04:13:40 2019
  OS/Arch:          linux/amd64
  Experimental:     true
$  DOCKER_BUILDKIT=1  docker build --tag testdocker --platform linux/ppc64le -f Dockerfile.platformargs .
$  docker create -ti --name container_test_docker testdocker file /bin/ls /tmp/ls
aec54aad0aa9ddba2a029da5107a199ef45e49c981965e4854c5286d9f8ee6fe
$ docker cp container_test_docker:/bin/ls /tmp/ubuntu_ls
$ docker cp container_test_docker:/tmp/ls /tmp/alpine_ls
$ file  /tmp/alpine_ls /tmp/ubuntu_ls
/tmp/alpine_ls: ELF 64-bit LSB pie executable, 64-bit PowerPC or cisco 7500, version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-powerpc64le.so.1, stripped
/tmp/ubuntu_ls: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=9567f9a28e66f4d7ec4baf31cfbf68d0410f0ae6, stripped

I would of assumed the final image would be of the target architecture.

The first image architecture, if there are multiple images is less obvious as to what is the choice. When you have a multiple FROM statements in a multi-stage image its fairly obvious that the last one is of the target. Earlier stages however are more ambiguous, they could be of the build (native) architecture to cross-compile some artefacts, or build architecture agnostic artefacts (which is faster not emulated), or they could be of the target architecture to have components.

I was going to test with docker buildx to see what behaviour it exhibited but its currently not recognising the buildx plugin install.

[tonistiigi]

» DOCKER_BUILDKIT=1  docker build --tag testdocker --platform linux/ppc64le -f Dockerfile .                                                                                                                                            !10050
[+] Building 9.0s (7/7) FINISHED
 => [internal] load build definition from Dockerfile                                                                                                                                                                                     0.0s
 => => transferring dockerfile: 35B                                                                                                                                                                                                      0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                        0.0s
 => => transferring context: 2B                                                                                                                                                                                                          0.0s
 => [internal] load metadata for docker.io/library/ubuntu:latest                                                                                                                                                                         1.5s
 => FROM docker.io/library/alpine:latest                                                                                                                                                                                                 0.9s
 => => resolve docker.io/library/alpine:latest                                                                                                                                                                                           1.0s
 => => sha256:774d1c824114747b62a96c94f3856fcbbc117ea204e801d43acfd7735b9c6b6c 2.81MB / 2.81MB                                                                                                                                           0.3s
 => => sha256:ca1c944a4f8486a153024d9965aafbe24f5723c1d5c02f4964c045a16d19dc54 1.64kB / 1.64kB                                                                                                                                           0.0s
 => => sha256:c4f32ede4ca905e0410490655b8993abb9fb77742e90a570ac28a8ca334f6c2b 528B / 528B                                                                                                                                               0.0s
 => => sha256:3196b138dac3130672969393c8ec5ab317154299b0ba3c7a0ff6f72251ba1303 1.51kB / 1.51kB                                                                                                                                           0.0s
 => => extracting sha256:774d1c824114747b62a96c94f3856fcbbc117ea204e801d43acfd7735b9c6b6c                                                                                                                                                0.5s
 => [stage-1 1/2] FROM docker.io/library/ubuntu@sha256:9b1702dcfe32c873a770a32cfd306dd7fc1c4fd134adfb783db68defc8894b3c                                                                                                                  7.1s
 => => resolve docker.io/library/ubuntu@sha256:9b1702dcfe32c873a770a32cfd306dd7fc1c4fd134adfb783db68defc8894b3c                                                                                                                          0.0s
 => => sha256:5fb5943989879ea71f39058451b18a9cfe7c8d63d5e98bea0f8ce8398b662cfd 35.19kB / 35.19kB                                                                                                                                         0.1s
 => => sha256:03367c790f847a4fb9da497ba924c531b08ed840ced5ddebd576b1e617fdeabc 856B / 856B                                                                                                                                               0.3s
 => => sha256:9b1702dcfe32c873a770a32cfd306dd7fc1c4fd134adfb783db68defc8894b3c 1.42kB / 1.42kB                                                                                                                                           0.0s
 => => sha256:b11b1467edefe08a02b4eb3799cb7df8f19a25f0b2da2ef837b993ad2c750c3c 1.15kB / 1.15kB                                                                                                                                           0.0s
 => => sha256:4dc4a19dc1271abc8daffce96513470c1cb3a909784bdc92f838bd4ffb6f92ac 3.41kB / 3.41kB                                                                                                                                           0.0s
 => => sha256:ef5b5b197566d2f3081c54f8e5ce315ef1bb409a622708e21f26f8a316324dc9 30.39MB / 30.39MB                                                                                                                                         1.5s
 => => sha256:7a0dfc04432356cee71730dc1476c35948bcaa233371a0bee11cf8be333b28da 189B / 189B                                                                                                                                               0.4s
 => => extracting sha256:ef5b5b197566d2f3081c54f8e5ce315ef1bb409a622708e21f26f8a316324dc9                                                                                                                                                3.8s
 => => extracting sha256:5fb5943989879ea71f39058451b18a9cfe7c8d63d5e98bea0f8ce8398b662cfd                                                                                                                                                0.1s
 => => extracting sha256:03367c790f847a4fb9da497ba924c531b08ed840ced5ddebd576b1e617fdeabc                                                                                                                                                0.0s
 => => extracting sha256:7a0dfc04432356cee71730dc1476c35948bcaa233371a0bee11cf8be333b28da                                                                                                                                                0.0s
 => [stage-1 2/2] COPY --from=alpine /bin/ls /tmp/ls                                                                                                                                                                                     0.3s
 => exporting to image                                                                                                                                                                                                                   0.0s
 => => exporting layers                                                                                                                                                                                                                  0.0s
 => => writing image sha256:f71f2e3f40097617014b0d0675332d973fb615ae8f8268a37d0b99653d065c82                                                                                                                                             0.0s
 => => naming to docker.io/library/testdocker                                                                                                                                                                                            0.0s

»  docker create -ti --name container_test_docker testdocker file /bin/ls /tmp/ls
8c0f7d98de80fbc639d4ec7904717110081f6ebf6fa8546a95bd832fb09dc366

» docker cp container_test_docker:/bin/ls /tmp/ubuntu_ls

» docker cp container_test_docker:/tmp/ls /tmp/alpine_ls

» file  /tmp/alpine_ls /tmp/ubuntu_ls
/tmp/alpine_ls: ELF 64-bit LSB pie executable 64-bit PowerPC or cisco 7500, version 1 (SYSV), dynamically linked, interpreter /lib/ld-, stripped
/tmp/ubuntu_ls: ELF 64-bit LSB pie executable 64-bit PowerPC or cisco 7500, version 1 (SYSV), dynamically linked, interpreter /lib64/l, for GNU/Linux 3.10.0, BuildID[sha1]=42226d7636226377aa2cf8e7644821a0cbeeb505, stripped

I think the difference is that you had the ubuntu image locally in docker images for x86 while you didn't have alpine. If image exists locally it is always used instead of accessing the registry. This isn't really obvious and hopefully can be fixed with containerd storage integration(atm we do not have platform information for local images). Currently, it should always work correctly if you do docker build --pull .

[grooverdan]

Thanks so much for the reproduction and explanation. You're undoubtedly right about ubuntu being local, I just haven't managed to remove it (even with --pull and/or --no-cache).

Is there a containerd issue tracking the lack of platform information? I couldn't see it in the resolved or open list.

[thaJeztah]

@grooverdan I think @tonistiigi is referring to moby/moby#38043 (which is a big change, but actively being worked on)