multi-arch alpine image support

[arm64b]

We need this definitely :-)

With that, you can:
'docker pull alpine:3.6' will get a x86_64 based docker image, while the same pull command will get an arm64-based docker image.

BTW,'libunwind-dev' pkg is missed from arm64 now...

[andyshinn]

I think this is in the works upstream. It may be something we can leverage when it is supported. But it first needs to be a Docker mechanism for detecting arch and switching the downloaded image on the fly.

According to the Fedora official image at https://github.com/docker-library/official-images/blob/master/library/fedora, this may be supported in the official library. I'm not quite sure what it means yet (if we would have to maintain all the arch or if they could be different repositories).

Either way, we also need some sort of build infra that can handle different arches. Right now we use CircleCI with is only amd64 AFAIK.

[tianon]

Couldn't we use the minirootfs tarballs upstream publishes? (see "MINI ROOT FILESYSTEM" on https://alpinelinux.org/downloads/)

That won't get us all the versions of Alpine supported here (nor edge), but it does cover at least 3.5 and 3.6 (for the supported arches of each).

As for official images, they can be in many repos, or a single repo, many branches or a single branch -- it's up to you. Here's the blurb I sent to @ncopa on the subject a little while ago (don't think anything was ever posted publicly, although I think we added you to the conversation at the time; probably got lost in the shuffle 😅):

we're finally officializing a lot of the multiarch work we've been doing, and as such I think we've finally got some decent examples that are worth looking at for how Alpine might be enabled for multiarch 🙂docker-library/official-images#3024 (ubuntu) and docker-library/official-images#3031 (debian) are our first two "base" images with multiarch enabled (no manifest lists yet, just official multiarch specification so we can build/push from a single source of truth -- manifest lists are on the TODO shortly though); I've set up https://github.com/tianon/docker-brew-alpine-multiarch as another example of how this could go down, using the minirootfs tarballs that have been officially released (with https://github.com/tianon/docker-brew-alpine-multiarch/blob/master/alpine being an example library file for it)

Since then, as you've noted, fedora has added this (which was very recent).

As for the Docker engine, it does have some preliminary multiarch support. Given a manifest list with proper platform specifications, it will pull an appropriate image for running based on the current engine's platform, and Swarm is even learning to schedule the appropriate images based on architecture. 😄

See docker-library/official-images#2289 for the tracking issue for official images + manifest lists. 👍

[andyshinn]

Can the official library just pull tarballs from the website or do they need to be in git?

[tianon]

Unfortunately they need to be in Git, although I'm happy to help automate that if you'd like!

[tianon]

I think we could use "date" and "time" from "latest-releases.yaml" to create reproducible commit hashes too.

[alexkreidler]

TLDR: I am working on a PR and would like input on the structure of our build artifacts

See my comments on #237 for initial thoughts and implementation details. I am trying to change as little code as possible but would like provide robust multi-arch support.

I've got the alpine-builder docker image able to pull the different architecture images supported by alpine into a rootfs.tar.xz.

Example

export RELEASE="v3.6"
export MIRROR="http://dl-cdn.alpinelinux.org/alpine"
export PACKAGES="alpine-baselayout,alpine-keys,apk-tools,libc-utils"
export ARCH="armhf"
export BUILD_OPTIONS=(-b -s -t UTC -r $RELEASE -m $MIRROR -p $PACKAGES -a $ARCH)
export TAGS=(alpine:3.6 alpine:latest)

However, this would require a separate folder and config for each architecture, like this:

 versions
    ├── gliderlabs-3.1-x86_64
    │   ├── Dockerfile
    │   └── options
    ...
    ├── library-3.6-x86-64
    │   ├── Dockerfile
    │   ├── options
    │   └── rootfs.tar.xz
    ├── library-3.6-aarch64
    │   ├── Dockerfile
    │   ├── options
    │   └── rootfs.tar.xz
    ...
    └── library-3.6-armhf
        ├── Dockerfile
        └── options

There would be an explosion in the number of folders, and it would be very difficult to maintain.
I am also opposed to generating multiple architectures in one run of the alpine-builder dockerfile.
One idea is to have another config variable, like TAGS, but for the architectures.

export RELEASE="v3.6"
export MIRROR="http://dl-cdn.alpinelinux.org/alpine"
export PACKAGES="alpine-baselayout,alpine-keys,apk-tools,libc-utils"
export ARCHS="x86_64,x86,armhf,aarch64"
export BUILD_OPTIONS=(-b -s -t UTC -r $RELEASE -m $MIRROR -p $PACKAGES)
export TAGS=(alpine:3.6 alpine:latest)

This would then just need an additional loop in the build.sh script to generate an image for each architecture of each release, and some additional code to generate the library spec file.

I'd love ideas on the best way to organize all this. Here is the debian structure for reference. I will update with a link to the WIP PR.

ping @andyshinn

[vielmetti]

@tim15 - can you give an update on the Alpine multiarch builds?

[alexkreidler]

@tianon has had unofficial multiarch support in Jenkins for a while, just by downloading the tarballs. For example, it generates an image like arm32v6/alpine. One limitation is that upstream has only recently started releasing the minirootfs tarballs, older versions (=<3.4) are not be supported. Also, it currently doesn't link the manifests because it the library spec files are generated separately, so it cannot link them.

We could just effectively replicate that in this official repository and generate the library spec files here.

Another option is to emulate the appropriate architecture, and run the scripts on that architecture. I'm attempted to set up some CircleCI builds for a modified version of qemu from resin-io/qemu so we can build all the architectures docker supports, but it's been difficult. See this post https://resin.io/blog/building-arm-containers-on-any-x86-machine-even-dockerhub/ for details. This would allow us to build the older versions as well (but why would we want to?)

We're trying to figure out the direction to head in, so I'd like input from @andyshinn and @tianon on this. Do you guys think we should emulate the different architectures, replicate + modify the Jenkins script here, or leave it pretty much as-is (maybe with some changes for manifest lists, etc)?

[andyshinn]

I'm still thinking about this and will be able to spend more time on it in October. But as a lot of work already went into making the building more multi-arch, one possibility would be moving to a CI solution that has Docker runners for multi-arch (or at least amd64 and arm). I haven't seen a service like this. But does one exist that we want to try?

[vielmetti]

If you need arm64 build infra I have some to offer from the @WorksOnArm project.

[justincormack]

I dont think there is a need to make the older images work; this is a new feature and only supporting 3.5 and 3.6 seems totally reasonable.

[andyshinn]

We discussed this a bit and think it makes sense to just pull the mini rootfs tarballs that are already at https://alpinelinux.org/downloads/ and store them. Here are my thoughts on how we would accomplish this incrementally while still supporting the current builder:

• Each $ARCH will get its own options file and the current options files move to $versions_dir/$arch (some of this work already done).
• Introduce a new options variable such as PULL_URL that preempts the builder and essentially ignore RELEASE, MIRROR, PACKAGES and BUILD_OPTIONS. It might look like PULL_URL=http://dl-cdn.alpinelinux.org/alpine/v3.6/releases/armhf/alpine-minirootfs-3.6.2-armhf.tar.gz. We could also re-use / add some variables to somewhat template the URL such as PULL_URL=http://dl-cdn.alpinelinux.org/alpine/$RELEASE/releases/$ARCH/alpine-minirootfs-$RELEASE.$MINOR-$ARCH.tar.gz.
• Split the build function into two (such as image_build and image_pull), one that handles the existing logic for building an image from the builder and one that just pulls the tarball and puts it in the correct place. Then build handles the switching logic based on if PULL_URL exists.
• The build script probably needs to be more compression agnostic (some xz stuff is hardcoded but the upstream tarballs are gzipped). Maybe we just rely on tar detecting the compression and fully qualify file names?
• We might need an additional options variable for mapping the correct $ARCH name to the Docker official library naming (for example, armhf in Alpine might be equal to armv6 in Docker library).
• I'm thinking we just support this for 3.5+ Docker library images only as a start (gliderlabs variants still use the builder and are arm64 only).

Names are just examples as I run through in my head. Obviously, I leave naming and implementation up to the one working on this. There may be other parts I'm missing to make this work as well.

@tim15 what do you think about this? I know you started some work on multi-arch in the builder already and this idea throws some of that out the window. But there are definitely bits that make sense and I'm grateful of you starting the work. Is it something you want to take a stab at?

[alexkreidler]

Thanks @andy for good ideas and direction. Sounds good! I'll take a stab at this and get back to you all with updates.

One additional idea is to auto-generate the options files from the latest-releases.yaml file:
http://dl-2.alpinelinux.org/alpine/v3.6/releases/armhf/latest-releases.yaml, but that will come later.

e.g.

---
[cut for brevity]
-
  title: "Mini root filesystem"
  desc: |
    Minimal root filesystem.
    For use in containers
    and minimal chroots.
  branch: v3.6
  arch: armhf
  version: 3.6.2
  flavor: alpine-minirootfs
  file: alpine-minirootfs-3.6.2-armhf.tar.gz
  iso: alpine-minirootfs-3.6.2-armhf.tar.gz
  date: 2017-06-17
  time: 09:47:35
  size: 1901056
  sha256: 14f62c60adf26996df8a5882604914f655920003f47d3a135e263cb31ce7cfe7
  sha512: 860b4a426b126626d154be1a5122a9d7d5909ac3cd7ae59e943182a2ad07eb9b41a28f861546c0d214f7027126fb0536e9ceeb0c1de46a336e39bf5290e5a20d

So then we could get the PULL_URL including MINOR version number automatically.

[friism]

@tim15 @andyshinn thanks for working on this!

(I work at Docker, see this post)

Because so many of our official images rely on Alpine, we're really looking forward to this update. Please let us know if there's any help we can offer. @tianon has led the rest of the multi-arch effort and would be available to assist if that's useful.

[tianon]

Friendly ping @tim15; any progress? Anything I can help with? 🙏 ❤️

(This is blocking a lot of multi-arch work ATM 😅 ❤️)

[alexkreidler]

Sure! You can check out what I have so far at https://github.com/Tim15/docker-alpine/tree/pull_tarballs. I guess I'll create a WIP PR and link to it here. Currently, we split up the build() function in the build file into pull_tarballs() and build_image(), with a should_pull() function to determine which to do. I am trying to maintain compatibilty with the older tags as @andyshinn has indicated. I am going to get build() working and then go on to rewrite library() to generate the spec file properly. Hopefully we'll finish this up soon!

[alexkreidler]

See #337.

[andyshinn]

We've completed the first release for multi-arch support which should be in now (or soon) for official library 3.6 and edge images.

[tianon]

🤘

$ manifest-tool inspect alpine:3.6
Name:   alpine:3.6 (Type: application/vnd.docker.distribution.manifest.list.v2+json)
Digest: sha256:b40e202395eaec699f2d0c5e01e6d6cb8e6b57d77c0e0221600cf0b5940cf3ab
 * Contains 6 manifest references:
1    Mfst Type: application/vnd.docker.distribution.manifest.v2+json
1       Digest: sha256:a09e6318d74f23fe68bad84259fedf5e6ce02c38537336419ee5e9abe10aa2af
1  Mfst Length: 528
1     Platform:
1           -      OS: linux
1           - OS Vers: 
1           - OS Feat: []
1           -    Arch: amd64
1           - Variant: 
1           - Feature: 
1     # Layers: 1
         layer 1: digest = sha256:b56ae66c29370df48e7377c8f9baa744a3958058a766793f821dadcb144a4647

2    Mfst Type: application/vnd.docker.distribution.manifest.v2+json
2       Digest: sha256:2dd7795cbac6f8d9a7780611802d292eb09cdbf4c569b22f88ed7b7124b7c12f
2  Mfst Length: 735
2     Platform:
2           -      OS: linux
2           - OS Vers: 
2           - OS Feat: []
2           -    Arch: arm
2           - Variant: v6
2           - Feature: 
2     # Layers: 2
         layer 1: digest = sha256:0864efeeb5cb8dca4eb53e5d6fd38486daee80fa326fe36d1ad254f8fa6bb310
         layer 2: digest = sha256:3cda69762aee1588fa82aeabf1af6d6ad24f737cce1451fab2e0199849b1e12e

3    Mfst Type: application/vnd.docker.distribution.manifest.v2+json
3       Digest: sha256:4d398eec2867d7be55d6916d91d99670ead7d4c9cdd72b21439fcbf71a18f73e
3  Mfst Length: 735
3     Platform:
3           -      OS: linux
3           - OS Vers: 
3           - OS Feat: []
3           -    Arch: arm64
3           - Variant: v8
3           - Feature: 
3     # Layers: 2
         layer 1: digest = sha256:bb473f0ebc12fde1bd45c1bd3c46f2d3aab367b1b7739464771455b9972f7894
         layer 2: digest = sha256:75ff6b7ff3a208b8399e701e7ea1b7edbdc654c8c60d33c6f09a7803e2dda776

4    Mfst Type: application/vnd.docker.distribution.manifest.v2+json
4       Digest: sha256:ee813612f9d7710d730c4574e7d835983bce17517165ddca0fb8f0b21856d780
4  Mfst Length: 735
4     Platform:
4           -      OS: linux
4           - OS Vers: 
4           - OS Feat: []
4           -    Arch: 386
4           - Variant: 
4           - Feature: 
4     # Layers: 2
         layer 1: digest = sha256:ffe4428ef008913a7ec63449e4ad3aa536b26103943146a302591dfceb157d2f
         layer 2: digest = sha256:5f4fe786260f2bd2710289e7c9487b423cb252a691fa501759b0768516122869

5    Mfst Type: application/vnd.docker.distribution.manifest.v2+json
5       Digest: sha256:adfb9b0b46b636098c67c2bca218c18e62048e439b9cf5a07e3668c8de96971d
5  Mfst Length: 735
5     Platform:
5           -      OS: linux
5           - OS Vers: 
5           - OS Feat: []
5           -    Arch: ppc64le
5           - Variant: 
5           - Feature: 
5     # Layers: 2
         layer 1: digest = sha256:1e52418956f7d2a8ea35e8e6e3318fd08e005b27457d77868c225e7433bbfa02
         layer 2: digest = sha256:acf472f4e5bb7956ac20bb343b304e1d3de1f79160c0d158cccbe25980022d50

6    Mfst Type: application/vnd.docker.distribution.manifest.v2+json
6       Digest: sha256:b20b3fc6e3959f9c761137fcf81f97bd0a36fe9f3908f24fda679df630b840d6
6  Mfst Length: 735
6     Platform:
6           -      OS: linux
6           - OS Vers: 
6           - OS Feat: []
6           -    Arch: s390x
6           - Variant: 
6           - Feature: 
6     # Layers: 2
         layer 1: digest = sha256:d45fd9d3c4f188ab1f3a4bf6a9f5202b3f1577dbb998f5f28e82d192e0c1f0e7
         layer 2: digest = sha256:0e5978b6b34b3e943e0fd25dfb50991c0bad82a986cfdaa91c4de756431ba679