Build arm64 main image #1487

felixvanoost · 2023-03-25T13:08:32Z

Builds an arm64 variant of the main Kroki Docker image by leveraging the changes introduced in #1476. All images should now natively support the ARM architecture. This includes the following changes:

Creates a nightly workflow to build the multi-arch images every night at 02:00 UTC.
Restricts the Docker Buildx builder to 2 parallel threads. This matches the 2 cores available on the GitHub CI runners and prevents timeouts during the build process.
Runs a free disk space action at the start of the CI workflow to recover extra disk space from the CI runner. This is now necessary to successfully complete the build process.

ggrossetie · 2023-03-25T14:07:50Z

Could you please rebase on main? Thanks!

felixvanoost · 2023-03-26T04:19:13Z

The images are building locally but I've been running into several issues with the CI:

The pipeline was initially reporting a Docker error 317 due to an issue with cargo install needing too long / too much memory to get the crates.io index as described in this issue. I resolved this by upgrading to the latest Rust release and enabling sparse-registry.
Next we starting getting some strange network issues with npm install, which I resolved by setting maxsockets=1 but is probably not doing the pipeline any favours in terms of speed.
Now, it's failing because of a timeout with many of the tests. No idea what's causing this but I will continue looking later.

The full pipeline now takes over an hour because it has to build both arm64 (using QEMU, which is slower than native) and amd64 builds. Not ideal, and will probably need to be resolved through more static images.

ggrossetie · 2023-03-26T09:58:01Z

The full pipeline now takes over an hour because it has to build both arm64 (using QEMU, which is slower than native) and amd64 builds. Not ideal, and will probably need to be resolved through more static images.

Indeed 🙀
However, I don't think we need to build both arm64 and amd64 on every commit. We could setup a nightly build (with both amd64 and arm64) and a CI build with only amd64.

The images are building locally but I've been running into several issues with the CI:

We were already close to the GitHub runner limit and I guess adding the arm64 build tipped the balance.
Anyway, thanks for finding workarounds and improvements 👍🏻

Also it might be possible to reduce the parallelization of buildx by running it multiple times on different targets:

docker buildx bake kroki-python
docker buildx bake kroki-js
docker buildx bake kroki-java

felixvanoost · 2023-04-01T15:53:32Z

I've defined a nightly CI workflow that will build the multi-arch images and also split up the docker buildx bake calls for the main vs. companion images to reduce parallelisation. All of this works fine locally, but we're still getting a timeout error for some of the tests defined in /vega/tests/test.js.

@ggrossetie I couldn't figure out where the 2000ms timeout is defined for these tests. Is there a global config somewhere? I think that the problem would be resolved by increasing the timeout slightly, because running the arm64 image on QEMU is slower than native.

ggrossetie · 2023-04-02T15:40:58Z

I couldn't figure out where the 2000ms timeout is defined for these tests. Is there a global config somewhere? I think that the problem would be resolved by increasing the timeout slightly, because running the arm64 image on QEMU is slower than native.

I believe this is the default value of Mocha but you can override it using this.timeout(15000) (15 seconds in milliseconds) inside the describe function in

kroki/vega/tests/test.js

Lines 14 to 15 in b039f17

    
           describe('#convert', function () { 
        
             it('should throw UnsafeIncludeError in secure mode when the Vega-Lite specification contains data.url', async function () {

felixvanoost · 2023-04-06T04:03:35Z

After resolving the parallelism issues (I think), I've now encountered issues with running out of disk space on the runners. For now I'm trying to run this action at the start of the workflow, and I'll see if that fixes things.

@ggrossetie Please let me know if you'd like me to stop this; I don't want to waste all your CI minutes. 🙃

felixvanoost · 2023-04-07T01:15:49Z

This is now good to go from my side. I've updated the PR description above with the main changes.

I tested both regular (native-arch) and multi-arch CI workflows manually in the two last commits. Nightly and release builds are multi-arch, regular builds are native-arch.

ggrossetie · 2023-04-14T15:28:54Z

@felixvanoost perfect, I will take a closer look this weekend 👍🏻

ggrossetie · 2023-04-16T10:17:03Z

I will try to build the arm64 image on an Ampere A1 Compute machine. I heard that QEMU is ~20 times slower.

ggrossetie · 2023-04-16T10:18:17Z

Also, in the next version we will use UMLet and PlantUML as native images (using GraalVM). So we need to produce arm64 compatible native images.

ggrossetie · 2023-04-16T13:28:03Z

.github/workflows/main.yaml

+    - name: Free up disk space
+      uses: jlumbroso/free-disk-space@main
+      with:
+        android: true
+        dotnet: true
+        haskell: true
+        large-packages: true
+        swap-storage: true


I think we should use a condition to use this action only if build_multiarch is true

ggrossetie · 2023-04-16T13:28:38Z

.github/workflows/main.yaml

    - name: Set up Docker Buildx
      uses: docker/setup-buildx-action@v2
+      with:
+        config-inline: |
+          [worker.oci]
+            max-parallelism = 2


Same here,we should set max-parallelism to 2 when build_multiarch is true

ggrossetie · 2023-04-16T14:24:28Z

Thanks for all your great work @felixvanoost 🙌🏻

felixvanoost · 2023-04-16T14:31:23Z

My pleasure!

Co-authored-by: Guillaume Grossetie <[email protected]>

felixvanoost · 2023-04-17T03:08:09Z

I will try to build the arm64 image on an Ampere A1 Compute machine. I heard that QEMU is ~20 times slower.

@ggrossetie A native arm64 runner would be great. I ended up using cross-compilation (see here) for most of the builds, so the CI build times should be much lower than they were initally.

QEMU is now only being used for D2, Pikchr, Mermaid, Wireviz, Blockdiag, and to put together the main Kroki image. Of those, D2, Pikchr, and Mermaid can most likely be cross-compiled with some additional effort.

ggrossetie · 2023-04-18T14:02:58Z

@felixvanoost I get the following errors when building images:

WARNING: local cache import at /tmp/.buildx-cache not found due to err: could not read /tmp/.buildx-cache/index.json: open /tmp/.buildx-cache/index.json: no such file or directory
WARNING: local cache import at /tmp/.buildx-cache not found due to err: could not read /tmp/.buildx-cache/index.json: open /tmp/.buildx-cache/index.json: no such file or directory
WARNING: local cache import at /tmp/.buildx-cache not found due to err: could not read /tmp/.buildx-cache/index.json: open /tmp/.buildx-cache/index.json: no such file or directory
WARNING: local cache import at /tmp/.buildx-cache not found due to err: could not read /tmp/.buildx-cache/index.json: open /tmp/.buildx-cache/index.json: no such file or directory
WARNING: local cache import at /tmp/.buildx-cache not found due to err: could not read /tmp/.buildx-cache/index.json: open /tmp/.buildx-cache/index.json: no such file or directory
WARNING: local cache import at /tmp/.buildx-cache not found due to err: could not read /tmp/.buildx-cache/index.json: open /tmp/.buildx-cache/index.json: no such file or directory
WARNING: local cache import at /tmp/.buildx-cache not found due to err: could not read /tmp/.buildx-cache/index.json: open /tmp/.buildx-cache/index.json: no such file or directory
ERROR: (*service).Write failed
ERROR: (*service).Write failed
ERROR: (*service).Write failed
ERROR: (*service).Write failed
ERROR: (*service).Write failed
ERROR: (*service).Write failed
ERROR: (*service).Write failed
ERROR: (*service).Write failed
ERROR: (*service).Write failed
ERROR: (*service).Write failed
ERROR: (*service).Write failed
ERROR: (*service).Write failed
ERROR: (*service).Write failed
ERROR: (*service).Write failed
ERROR: (*service).Write failed
ERROR: (*service).Write failed

Not sure if it means that the disk is full or if the latest version of buildx has a bug...? Does this error sound familiar?

ggrossetie · 2023-04-18T19:23:15Z

Apparently, when something is failing it throws this error but this is not the root cause

felixvanoost · 2023-04-19T01:44:25Z

@felixvanoost I get the following errors when building images:

WARNING: local cache import at /tmp/.buildx-cache not found due to err: could not read /tmp/.buildx-cache/index.json: open /tmp/.buildx-cache/index.json: no such file or directory
WARNING: local cache import at /tmp/.buildx-cache not found due to err: could not read /tmp/.buildx-cache/index.json: open /tmp/.buildx-cache/index.json: no such file or directory
WARNING: local cache import at /tmp/.buildx-cache not found due to err: could not read /tmp/.buildx-cache/index.json: open /tmp/.buildx-cache/index.json: no such file or directory
WARNING: local cache import at /tmp/.buildx-cache not found due to err: could not read /tmp/.buildx-cache/index.json: open /tmp/.buildx-cache/index.json: no such file or directory
WARNING: local cache import at /tmp/.buildx-cache not found due to err: could not read /tmp/.buildx-cache/index.json: open /tmp/.buildx-cache/index.json: no such file or directory
WARNING: local cache import at /tmp/.buildx-cache not found due to err: could not read /tmp/.buildx-cache/index.json: open /tmp/.buildx-cache/index.json: no such file or directory
WARNING: local cache import at /tmp/.buildx-cache not found due to err: could not read /tmp/.buildx-cache/index.json: open /tmp/.buildx-cache/index.json: no such file or directory
ERROR: (*service).Write failed
ERROR: (*service).Write failed
ERROR: (*service).Write failed
ERROR: (*service).Write failed
ERROR: (*service).Write failed
ERROR: (*service).Write failed
ERROR: (*service).Write failed
ERROR: (*service).Write failed
ERROR: (*service).Write failed
ERROR: (*service).Write failed
ERROR: (*service).Write failed
ERROR: (*service).Write failed
ERROR: (*service).Write failed
ERROR: (*service).Write failed
ERROR: (*service).Write failed
ERROR: (*service).Write failed

Not sure if it means that the disk is full or if the latest version of buildx has a bug...? Does this error sound familiar?

I've never experienced this before. Is this happening directly in the workflow?

Update: The latest nightly workflow just passed here.

mariotoffia · 2023-04-25T05:07:28Z

@felixvanoost thanks for you're porting kroki (of which I'm ever grateful @ggrossetie) to arm64!!

Cheers,
Mario :)

felixvanoost force-pushed the build-arm64-main-image branch from c2364ad to 5e9964b Compare March 25, 2023 22:26

felixvanoost added 7 commits March 31, 2023 18:53

Build arm64 version of Kroki main image

0ee1bf9

Map target architecture for Rust

91bbe9a

Update Rust arm64 mapping

595d89b

Attempt to build only arm64 image for debugging

384a213

Attempt to use Cargo sparse registry

66900cf

Don't specify architecture for SVGBob

2b41133

Restrict number of sockets for npm

a161e3a

felixvanoost force-pushed the build-arm64-main-image branch from d36bb7b to a161e3a Compare March 31, 2023 22:53

felixvanoost added 6 commits March 31, 2023 20:45

Build images for native architecture by default

458baa3

Attempt to build multi-arch containers in nightly CI workflow

a649590

Fix path issue

2ed0c35

Specify images to run smoke tests on

0b115ea

Test multi-arch builds for debugging

1f58029

Restrict npm to 2 concurrent sockets

b3fd2ce

felixvanoost added 6 commits April 2, 2023 16:02

Set Vega tests timeout to 15s

4d04c00

Prune images in between builds

3563f14

Merge branch 'yuzutech:main' into build-arm64-main-image

3be18f4

Restrict buildx parallelism to 2

f2ea43e

Don't prune images after building

ec8ffba

Free up runner disk space before workflows

8de9820

felixvanoost added 2 commits April 6, 2023 18:06

Clean up redundant function calls

7f4ad6f

FINALLY

6a2f010

felixvanoost mentioned this pull request Apr 12, 2023

Build arm64 Docker image webgiss/niolesk#37

Merged

Cross-compile when possible

bc3328a

This was referenced Apr 14, 2023

Docker containers for more platforms #1084

Closed

[question] Can kroki run on a Macbook M1 Pro? #1110

Open

ggrossetie reviewed Apr 16, 2023

View reviewed changes

ggrossetie added 4 commits April 16, 2023 15:37

Add condition to skip actions when build_multiarch is not enabled

7fa3903

Add a default value to setup buildx

dcacbd4

Global input with a default value

0e74dd8

Simplify syntax

8784264

ggrossetie merged commit 9cd4a45 into yuzutech:main Apr 16, 2023

ggrossetie added a commit that referenced this pull request Apr 16, 2023

Build arm64 main image (#1487)

f448164

Co-authored-by: Guillaume Grossetie <[email protected]>

felixvanoost deleted the build-arm64-main-image branch April 17, 2023 03:10

felixvanoost mentioned this pull request Apr 24, 2023

Install arm64-compatible Node packages #1523

Merged

lipkau mentioned this pull request May 3, 2023

docker hub is missing manifest for linux/arm64/v8 #1533

Closed

felixvanoost mentioned this pull request May 19, 2023

Exec format error for BPMN, Diagrams.net, and Excalidraw on arm64 #1547

Closed

felixvanoost mentioned this pull request Oct 25, 2023

TikZ not working on arm64 #1662

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build arm64 main image #1487

Build arm64 main image #1487

felixvanoost commented Mar 25, 2023 •

edited

Loading

ggrossetie commented Mar 25, 2023

felixvanoost commented Mar 26, 2023

ggrossetie commented Mar 26, 2023

felixvanoost commented Apr 1, 2023

ggrossetie commented Apr 2, 2023 •

edited

Loading

felixvanoost commented Apr 6, 2023

felixvanoost commented Apr 7, 2023

ggrossetie commented Apr 14, 2023

ggrossetie commented Apr 16, 2023

ggrossetie commented Apr 16, 2023

ggrossetie Apr 16, 2023

ggrossetie Apr 16, 2023

ggrossetie commented Apr 16, 2023

felixvanoost commented Apr 16, 2023

felixvanoost commented Apr 17, 2023 •

edited

Loading

ggrossetie commented Apr 18, 2023

ggrossetie commented Apr 18, 2023

felixvanoost commented Apr 19, 2023 •

edited

Loading

mariotoffia commented Apr 25, 2023

Build arm64 main image #1487

Build arm64 main image #1487

Conversation

felixvanoost commented Mar 25, 2023 • edited Loading

ggrossetie commented Mar 25, 2023

felixvanoost commented Mar 26, 2023

ggrossetie commented Mar 26, 2023

felixvanoost commented Apr 1, 2023

ggrossetie commented Apr 2, 2023 • edited Loading

felixvanoost commented Apr 6, 2023

felixvanoost commented Apr 7, 2023

ggrossetie commented Apr 14, 2023

ggrossetie commented Apr 16, 2023

ggrossetie commented Apr 16, 2023

ggrossetie Apr 16, 2023

Choose a reason for hiding this comment

ggrossetie Apr 16, 2023

Choose a reason for hiding this comment

ggrossetie commented Apr 16, 2023

felixvanoost commented Apr 16, 2023

felixvanoost commented Apr 17, 2023 • edited Loading

ggrossetie commented Apr 18, 2023

ggrossetie commented Apr 18, 2023

felixvanoost commented Apr 19, 2023 • edited Loading

mariotoffia commented Apr 25, 2023

felixvanoost commented Mar 25, 2023 •

edited

Loading

ggrossetie commented Apr 2, 2023 •

edited

Loading

felixvanoost commented Apr 17, 2023 •

edited

Loading

felixvanoost commented Apr 19, 2023 •

edited

Loading