baseImages: Updating base image versions

[garethtdavies]

@nghiant2710 I've manually updated the base image page for updated version information and to get a better sense of how it works before trying to pull this information in dynamically from the contracts.

I wrote a simple script to parse the version field for each variant from here for Node.js for example https://github.com/balena-io/contracts/blob/master/contracts/sw.stack/node/contract.json. A couple of questions as result:

• There are some images that do not have contracts but do exist in DockerHub, specifically Node 10.10 which is referenced in these docs https://hub.docker.com/layers/balenalib/raspberrypi3-debian-node/10.10.0-stretch-run/images/sha256-15c6d3683496a4813d487e79796f30debf1a637a42ebca258c3b6224ea15784f. As I understand these point to the corresponding frozen date image. So only those with contracts get updates?
• As per Update base images to include Raspbian #1254 there is a Raspbian image but again doesn't have a contract as I would expect under
  sw.os https://github.com/balena-io/contracts/tree/master/contracts/sw.os. Is there support for Raspbian?

Change-type: patch
Signed-off-by: Gareth Davies [email protected]

[nghiant2710]

Great work @garethtdavies 👍

Regarding your questions:

1. We only keep the latest version in contracts so you won't find information about old version there.
2. We do support raspbian and all devices with arch.sw = rpi will be built off balenalib/rpi-raspbian. As the raspbian images are not available on dockerhub, we create the balenalib/rpi-raspbian from this repo https://github.com/balena-io-library/resin-rpi-raspbian and all base images of raspberry-pi device type (RPi 1 and Zero) will be Raspbian. For some reasons discussed long time ago, we want to have raspberry-pi-debian as an alias of rpi-raspbian so you will see rpi in debian contract here https://github.com/balena-io/contracts/blob/master/contracts/sw.os/debian/contract.json

[lurch]

I wonder if it's also worth having a separate Python3 language tag, bearing in mind the upcoming EOL of Python2 ? https://pythonclock.org/

[lurch]

There's also part of the page which says "latest points to the latest Python 2 version, which currently is 2.7.17" and I guess that means there's currently no way to automatically get the latest Python 3 version?

[garethtdavies]

I missed this one. I'd have thought using 3 would do it but that doesn't seem to work as expected:

docker run -it balenalib/raspberrypi4-64-alpine-python:3-3.11-run bash
bash-5.0# python --version
Python 3.6.9

[garethtdavies]

Maybe that's the default package available i.e via apt-get install python3? I don't think that is correct and I don't know why it would default to that version when 3.8 is available https://github.com/balena-io-library/base-images/tree/master/balena-base-images/python/raspberrypi4-64/alpine/3.11

[lurch]

Created balena-io-library/base-images#606 to track this inconsistency

[garethtdavies]

Thanks for this explanation @nghiant2710 . I've tested this quite a bit now, but I am still a bit confused about Raspbian and how to document it.

I understand those device types with device type raspberry-pi will use Raspbian by default, but later variants, notably, the Pi3 does not. If I use the machine name raspberrypi3 or %%BALENA_MACHINE_NAME%% I am always going to get Debian if I don't specify an OS. I can specify Ubuntu, Fedora, and Alpine but not Raspbian, so to get Raspbian on the Pi3 I can get it e.g.:

FROM balenalib/raspberry-pi (i.e. a device type of raspberry pi)
FROM balenalib/rpi-raspbian:jessie
FROM balenalib/raspberry-pi-debian

I guess I'm expecting these to work, but they, of course, do not:

FROM balenalib/raspberrypi3-raspbian
FROM balenalib/%%BALENA_MACHINE_NAME%%-raspbian

So, I'm not sure how we should document this for Raspbian. Basically, if you want to use Raspbian, use the base image balenalib/rpi-raspbian with available tags of jessie, stretch and buster? (I also tested this on 32 and 64-bit host OS and worked on both)

[xginn8]

small nits, otherwise lgtm

[lurch]

In case it helps to clear up any confusion (at least from the Raspbian side, if not from the Balena side 😉 ) - the Raspbian images available from https://www.raspberrypi.org/downloads/raspbian/ support all models of the Raspberry Pi (Pi1, Pi Zero, Pi2, Pi3, Pi4) with a single image (which is presumably why there's no raspberrypi3-raspbian), and it's a 32-bit kernel and 32-bit userland.

I dunno how closely Balena's raspbian images follow the Raspberry Pi raspbian images though.

[nghiant2710]

@garethtdavies, you're right, only raspberry-pi device type is raspbian, as you can run armv7 debian images on Pi2 so we use it as the base images for Pi2 and Pi3. However, we do add raspbian package source and pre-install userland in the Pi2 and Pi3 base images (basically armv7hf debian will raspbian userland installed).

Another thing is debian is the default OS for the base images, if you don't specify OS in the base images then you will get debian (or raspbian in case of raspberry-pi).

[garethtdavies]

Thanks all for the discussion, that's much clearer to me now 😄 At the root of this is simply what we should include in the docs with reference to Raspbian. To that end, I've added a note to the docs about Raspbian. @nghiant2710 can you review this note https://github.com/balena-io/docs/pull/1285/files#diff-aa7d2966be46f689a147702028e9ac59R129 or have any other suggestions about any other missing information or where this should be included?

I've also updated this file for the latest contracts, and I have done a general cleanup, but I'll follow up with additional PRs for auto-generating these docs.

[lurch]

Should this mention that latest can also be used for the <distro_ver> tags?

[lurch]

So I guess if you specify a <lang_ver> but not a <lang_stack>, the lang_ver will just be ignored?

[garethtdavies]

It won't be ignored and it'd probably just mean the image doesn't exist (unless it happened to coincide with an available tag) - but either way, it's unlikely to be what you want. For example, you can see the raspberrypi3-alpine tags here https://hub.docker.com/r/balenalib/raspberrypi3-alpine/tags so if you specified a <lang_ver> tag of 10.8 (e.g. node) then there simply isn't a corresponding image.

[lurch]

Cool, I was just checking what the edge-case behaviour was 🙂

[lurch]

Hmmm, that's curious - https://hub.docker.com/layers/balenalib/raspberrypi3-alpine/latest/images/sha256-054582a87ef96ba2243334500bb580246dba066967e9f6ad0c3bee655fde281d says "LABEL io.balena.device-type=raspberry-pi2" ??

[lurch]

Presumably in both this table and the next, the "Default" column is the version you'd get if you used the latest tag for the <distro_ver> or <lang_ver> fields?
Is it worth mentioning that explicitly? (e.g. by changing the "Default" column-header to "Default (latest)" ? ) 🤷‍♂️

---

Hmmm, in the "Multiple Distributions:" and "Multiple language stacks:" lists earlier on in the page, I wonder if it'd be worth highlighting the versions corresponding to the "latest" tag by e.g. displaying them in bold? Like:

• Node.js: 8.11.1, 8.17.0, 9.11.2, 10.18.0, 11.15.0, 12.14.0 and 13.5.0
• Python: 2.7.17, 3.4.10, 3.5.7, 3.6.9, 3.7.5 and 3.8.0

[lurch]

The "Multiple language stacks:" list further up the page lists "Node.js 10.18.0" but here you're using a <lang_ver> of 10.18. Does that imply that balenalib/raspberrypi3-node:10.18, balenalib/raspberrypi3-node:10.18.0 and balenalib/raspberrypi3-node:10 are all equally valid base-image names? Or is there some other logic at work?

Possibly worthy of more explanation? 🤷‍♂️

[garethtdavies]

Yes. You can confirm this via docker pull balenalib/raspberrypi3-node:10.18 (and other tags) and inspecting the output of docker images which shows they are all the same image (for this specific example). As I noted earlier I think a few more examples would go a long way for this sort of information.

➜ ~ docker images
REPOSITORY                                                         TAG                   IMAGE ID            CREATED             SIZE
balenalib/raspberrypi3-node                                        10                    3bf113fa9176        15 hours ago        231MB
balenalib/raspberrypi3-node                                        10.18                 3bf113fa9176        15 hours ago        231MB
balenalib/raspberrypi3-node                                        10.18.0               3bf113fa9176        15 hours ago        231MB

[lurch]

Hmmm, I wonder if this might explain the behaviour you discovered at #1285 (comment) where balenalib/raspberrypi4-64-alpine-python:3-3.11-run points to an image containing Python 3.6.9 (whereas you'd expect it to contain Python 3.8.0) ...

Perhaps when python:3.6.9 got added, it also added the python:3.6 and python:3 tags, inadvertently "overwriting" the python:3 tag which was previously pointing to python:3.8.0?

That's just a guess though, and only @nghiant2710 will know for sure.

[lurch]

Hmmm, I've just noticed that this table says that the Default version of Alpine is v3.10, and by comparing Digests on https://hub.docker.com/r/balenalib/raspberrypi3-alpine/tags you can also confirm that the latest and 3.10 tags point to the same image, and yet Alpine 3.11 is mentioned in the "Multiple Distributions:" list on this page, and there's a 3.11 tag on DockerHub? 😕

[lurch]

Maybe that's explained by balena-io/contracts@84035e0#r36690619 but that's a bit of a random stab in the dark 😉

[garethtdavies]

Yeah looks like a mistake as mentioned in that linked contract above. In general though latest doesn't necessarily mean "latest" 😕https://medium.com/@mccode/the-misunderstood-docker-tag-latest-af3babfd6375 In this case though it looks like it should have been tagged latest.

[lurch]

That's interesting, but I've no idea how and/or if Docker tags relate to the base-image tags used by Balena 😉

[garethtdavies]

I'm not sure I understand that comment. The tags on Docker Hub are the base-image tags.

[lurch]

The medium.com article you've linked to mentions tags like marc/test:latest and it seems like the latest tag there is automatically added by Docker. I was assuming that that's actually unrelated to the latest tag embedded within a Balena image-name like balenalib/raspberrypi3-alpine-python:latest-3.7 ? 🤷‍♂️ 🤷‍♂️

[garethtdavies]

Squashed and rebased. I'm keen to get this in, in its current state, as the current docs are outdated and this fixes that. I'll take some of the unresolved suggestions to a follow-up PR as I'll likely need to tweak the layout to generate this doc from the contracts anyway.

[lurch]

Apologies if my comments / questions have delayed this PR, but hopefully they were more of a help than a hindrance.

[garethtdavies]

Oh no @lurch they have been super helpful and have helped clarify a lot of things. I didn't just want to merge this in under the assumption everything was resolved r.e. discussions above, so the note to acknowledge that. We can carry on the discussion but I don't want to delay getting at least the updated version information in 👍