Failed to build on Pi Zero W

[trilbytim]

I attempted to load this onto a Raspberry Pi Zero W. It seemed to get almost all the way through, but in the end failed to build Mosquitto, Influx DB, Node Rod or Grafana. Only Jupyter-lab was successfully built. I've attached the terminal output below.

MING stack install.txt

[ajlennon]

Very interesting - it looks like something may have gone missing upstream. I'll take a look!

[MatthewCroughan]

Not all of the images that we're using support Armv6, but it should work for some of the containers, just not Grafana last I checked. The real cause of the issue I think is this commit DynamicDevices/ming@3ee6b7c

My repo usesFROM --platform=linux/arm due to the Balena cloud builder being unable to automatically discover the arch for the device, or at least I was. Remember, this doesn't work when you use %%BALENA_ARCH%% as already explained in #24

This repo does not work at all at the moment for any device other than X86 because of that commit, so try mine, or rollback from that commit.

[MatthewCroughan]

Okay, that's even more hilarious. Seems Balena have started to call the %%BALENA_ARCH%% for the Pi 1 simply pi

From the log file provided:
Error: no matching manifest for linux/rpi in the manifest list entries

Can we please correct 3ee6b7c and get back to hardcoding this? There is no alternative, other than using BalenaOS docker base images, which is the reason the Jupyterlab container still built, something I created since there was no upstream image to choose from. Which defeats the whole purpose of using Docker and using things maintained by others, meaning we would have to maintain the entirety of Grafana and its installation method, dependencies, etc, rather than just its configuration with other software. Which I don't think we're getting paid enough for :)

[MatthewCroughan]

https://www.balena.io/docs/learn/develop/dockerfile/ How and why!

Device Name	BALENA_MACHINE_NAME	BALENA_ARCH
Raspberry Pi (1, Zero, Zero W)	raspberry-pi	rpi

[ajlennon]

They broke stuff by renaming things? Can they unbreak it then?

[MatthewCroughan]

@ajlennon No, it never worked in the first place. They have just made it more silly. Or maybe it was always this way, and we just never bothered to look at how silly it got. The only thing that matches up is X86.

armv7hf does not exist according to dockerhub, only armv7
amd64 does exist according to the dockerhub, and this is what it is called in %%BALENA_ARCH%%

It has only ever worked for x86 images, as I've already discussed in #24

[MatthewCroughan]

Reliance on %%BALENA_ARCH%% just hopes that they call it the same thing as the Docker Hub, which they don't, except in the case of amd64, so it's pointless. No doubt all of their CI relies on this, so we can't ask them to change it.

[ajlennon]

I'm pretty sure when I created the Dockerfiles this worked on arm?

[MatthewCroughan]

No they did not. It failed live during my talk, if you remember?

[MatthewCroughan]

The reason it failed during my talk on Balena at DoES is because the manifest armv7hf doesn't exist, because Balena call armv7 armv7hf. Which isn't what official Docker uses to describe their multi-arch image manifests.

[ajlennon]

Yeah but when I created the Dockerfiles it did work as I recall, which was a while before your talk?

[MatthewCroughan]

Yes, but we've changed through many PRs since then, I wrote at length about why here. #8

This doesn't change the problem introduced by relying on %%BALENA_ARCH%%, we need to hardcode the platform, unless we base ourselves on Balena OS platform dependant images.

If we do, not only does that mean we can't support both Docker and Balena independantly, but it also means that we have a crap ton more work to do, maintaining our own non standard, non supported, non official images, which I've already written about in #8

[MatthewCroughan]

I've asked a lot of people in the Docker Freenode IRC whether it's possible to change the FROM directive based on ENV or ARG and it seems that is not possible at all. There is no way to make our own list of %%BALENA_ARCH%% style variables, I mean.

Because Balena itself is an outdated version of Moby, the Docker Container framework, it does not support image manifests, again as I have already cross-referenced in the few issues that are in this repository countless times and is mentioned also in #8 extensively.

Here's more references for good measure.
https://forums.balena.io/t/balena-cloud-cli-builder-is-still-not-able-to-interpret-image-manifests/43203
moby/moby#36552
https://github.com/DynamicDevices/ming/issues/6

They have since privatised the repo responsible for the first comment in this issue.
balena-io/balena-cli#1508

As a result, you have to hardcode the platform argument, there is no alternative until they support mult-arch image manifests. Which has been an outstanding issue since 2018