Skip to content

Commit

Permalink
[ci] Add tests for non-native architectures
Browse files Browse the repository at this point in the history
This change introduces a new job to the CI to test the SDK on non-native
architectures. Github CI ATM provides amd64 runners by default, so the
new job targets testing on arch64 for now.

The tests are run in containers running on the target platform(s). The
container definitions are supposed to exist in the directory
.github/containers/nox-cross-arch. A definition for an
arm64-ubuntu20.04-python3.11 container can be found there.

Since the tests take quite a while to run, given that they run on
emulations, this step runs only on the `merge_group` event.

Signed-off-by: Tiyash Basu <[email protected]>
  • Loading branch information
tiyash-basu-frequenz committed Sep 5, 2023
1 parent a8f3d62 commit ec6ecd6
Show file tree
Hide file tree
Showing 7 changed files with 144 additions and 0 deletions.
21 changes: 21 additions & 0 deletions .github/containers/nox-cross-arch/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Cross-Arch Testing Containers

This directory containers dockerfiles that can be used in the CI to test the
python package in non-native machine architectures, e.g., `aarch64`.

The dockerfiles here follow a naming scheme so that they can be easily used in
build matrices in the CI, in `nox-cross-arch` job. The naming scheme is:

```
<arch>-<os>-python-<python-version>.Dockerfile
```

E.g.,

```
arm64-ubuntu-20.04-python-3.11.Dockerfile
```

If a dockerfile for your desired target architecture, OS, and python version
does not exist here, please add one before proceeding to add your options to the
test matrix.
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
FROM docker.io/library/ubuntu:20.04

ENV DEBIAN_FRONTEND=noninteractive

# Install Python 3.11 and curl to install pip later
RUN apt-get update -y && \
apt-get install --no-install-recommends -y \
software-properties-common && \
add-apt-repository ppa:deadsnakes/ppa && \
apt-get install --no-install-recommends -y \
ca-certificates \
curl \
git \
python3.11 \
python3.11-distutils && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*

# Install pip
RUN curl -sS https://bootstrap.pypa.io/get-pip.py | python3.11

RUN update-alternatives --install \
/usr/local/bin/python python /usr/bin/python3.11 1 && \
python -m pip install --upgrade --no-cache-dir pip

COPY entrypoint.bash /usr/bin/entrypoint.bash

ENTRYPOINT ["/usr/bin/entrypoint.bash"]
9 changes: 9 additions & 0 deletions .github/containers/nox-cross-arch/entrypoint.bash
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
#!/bin/bash
set -e

echo "System details:" $(uname -a)
echo "Machine:" $(uname -m)

pip install -e .[dev-noxfile]

exec "$@"
80 changes: 80 additions & 0 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,86 @@ jobs:
run: nox -R -e "$NOX_SESSION"
timeout-minutes: 10

nox-cross-arch:
name: Cross-arch tests with nox
if: github.event_name == 'merge_group'
strategy:
fail-fast: false
# Before adding new items to this matrix, make sure that a dockerfile
# exists for the combination of items in the matrix.
# Refer to .github/containers/nox-cross-arch/README.md to learn how to
# add and name new dockerfiles.
matrix:
arch:
- arm64
os:
- ubuntu-20.04
python:
- "3.11"
nox-session:
- "pytest_min"
runs-on: ${{ matrix.os }}

steps:
- name: Fetch sources
uses: actions/checkout@v3

- name: Set up QEMU
uses: docker/setup-qemu-action@v2
with:
platforms: linux/${{ matrix.arch }}

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2

# This is a workaround to prevent the cache from growing indefinitely.
# https://docs.docker.com/build/ci/github-actions/cache/#local-cache
# https://github.com/docker/build-push-action/issues/252
# https://github.com/moby/buildkit/issues/1896
- name: Cache container layers
uses: actions/cache@v3
with:
path: /tmp/.buildx-cache
key: ${{ runner.os }}-buildx-nox-${{ matrix.arch }}-${{ matrix.os }}-${{ matrix.python }}

- name: Build image
uses: docker/build-push-action@v4
with:
context: .github/containers/nox-cross-arch
file: .github/containers/nox-cross-arch/${{ matrix.arch }}-${{ matrix.os }}-python-${{ matrix.python }}.Dockerfile
platforms: linux/${{ matrix.arch }}
tags: localhost/nox-cross-arch:latest
push: false
load: true
cache-from: type=local,src=/tmp/.buildx-cache
cache-to: type=local,dest=/tmp/.buildx-cache-new,mode=max

# Refer to the workaround mentioned above
- name: Move cache
run: |
rm -rf /tmp/.buildx-cache
mv /tmp/.buildx-cache-new /tmp/.buildx-cache
# Cache nox downloads
- name: Cache nox downloads
uses: actions/cache@v3
with:
path: |
.nox/${{ matrix.nox-session }}
key: nox-${{ matrix.nox-session }}-${{ matrix.arch }}-${{ matrix.os }}-${{ matrix.python }}-${{ hashFiles('**/pyproject.toml') }}

- name: Run nox
run: |
docker run \
--rm \
-v $(pwd):/${{ github.workspace }} \
-w ${{ github.workspace }} \
--net=host \
--platform linux/${{ matrix.arch }} \
localhost/nox-cross-arch:latest \
nox -R -e ${{ matrix.nox-session }}
timeout-minutes: 30

build:
name: Build distribution packages
runs-on: ubuntu-20.04
Expand Down
4 changes: 4 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,10 @@ That said, if you want to test the actual website in **your fork**, you can
always use `mike deploy --push --remote your-fork-remote`, and then access the
GitHub pages produced for your fork.

## Cross-Arch Testing

This project has built-in support for testing across multiple architectures. Currently, our CI conducts tests on `aarch64` machines using QEMU emulation. We also have the flexibility to expand this support to include additional architectures in the future. For more information, see [Cross-Arch Testing](.github/containers/nox-cross-arch/README.md).

## Releasing

These are the steps to create a new release:
Expand Down
1 change: 1 addition & 0 deletions RELEASE_NOTES.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@

- A new class `Fuse` has been added to represent fuses. This class has a member variable `max_current` which represents the maximum current that can course through the fuse. If the current flowing through a fuse is greater than this limit, then the fuse will break the circuit.

- The CI now runs cross-arch tests on `aarch64` architectures.

## Bug Fixes

Expand Down
1 change: 1 addition & 0 deletions docs/.github/containers/nox-cross-arch/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
--8<-- ".github/containers/nox-cross-arch/README.md"

0 comments on commit ec6ecd6

Please sign in to comment.