Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable Multi-Arch Building and Pushing for registry viewer #125

Merged
merged 9 commits into from
Jun 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 24 additions & 2 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,23 @@ jobs:
uses: actions/checkout@c85c95e3d7251135ab7dc9ce3241c5835cc595a9 # v3.5.3
with:
fetch-depth: 0

- name: Free Disk Space
uses: jlumbroso/free-disk-space@54081f138730dfa15788a46383842cd2f914a1be #v1.3.1
with:
tool-cache: false
android: true
dotnet: true
haskell: true
large-packages: true
docker-images: true
swap-storage: true

- name: Set up QEMU # Enables arm64 image building
uses: docker/setup-qemu-action@68827325e0b33c7199eb31dd4e31fbe9023e06e3 #v3.0.0

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@d70bba72b1f3fd22344832f00baa16ece964efeb #v3.3.0

- uses: actions/setup-node@8f152de45cc393bb48ce5d89d36b731f54556e65 #v4.0.0
with:
Expand All @@ -111,9 +128,14 @@ jobs:
run: |
yarn install --frozen-lockfile

- name: Docker Build
- name: Docker Build for linux/amd64
run: |
yarn nx affected --target=docker-build --parallel=3
yarn nx affected --target=docker-build-amd64 --parallel=3

- name: Docker Build for linux/arm64
run: |
yarn nx affected --target=docker-build-arm64 --parallel=3


e2e:
name: E2E
Expand Down
10 changes: 6 additions & 4 deletions .github/workflows/pushimage-next.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,16 +31,18 @@ jobs:
steps:
- name: Check out devfile web source code
uses: actions/checkout@c85c95e3d7251135ab7dc9ce3241c5835cc595a9 # v3.5.3
- name: Set up QEMU # Enables arm64 image building
uses: docker/setup-qemu-action@68827325e0b33c7199eb31dd4e31fbe9023e06e3 #v3.0.0
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@d70bba72b1f3fd22344832f00baa16ece964efeb #v3.3.0
- name: Login to Quay
uses: docker/login-action@465a07811f14bebb1938fbed4728c6a1ff8901fc # v2.2.0
with:
registry: quay.io
username: ${{ secrets.QUAY_USERNAME }}
password: ${{ secrets.QUAY_PASSWORD }}
- name: Build the registry viewer image
run: bash ./scripts/build_viewer.sh
- name: Push the registry viewer image
run: bash ./scripts/push.sh registry-viewer:latest quay.io/devfile/registry-viewer:next
- name: Build and push the registry viewer image
run: bash ./scripts/build_multi_arch.sh
dispatch:
needs: registry-viewer-build
strategy:
Expand Down
8 changes: 7 additions & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,12 @@ ARG NEXT_PUBLIC_DOCSEARCH_APP_ID
ARG NEXT_PUBLIC_DOCSEARCH_API_KEY
ARG NEXT_PUBLIC_DOCSEARCH_INDEX_NAME

# Building different architectures via emulation is slow with Yarn
# This increases the timeout period so it can properly download dependencies
# Value is in milliseconds and is set to 60 minutes
# To increase/decrease you can override via --build-arg YARN_TIMEOUT=x in your build command
ARG YARN_TIMEOUT=3600000
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean a build might take close to one hour when this action runs?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The longest I've observed so far is around 30 minutes, I just increased it so we have a buffer since it is known to take a while. If GitHub supports arm64 runners in the future (the ones we use and not the large runners in which they now support arm), we'd be able to build the arm64 image in minutes.


# Check if the PROJECT_NAME build argument is set
RUN \
if [ "$PROJECT_NAME" == "landing-page" ] || [ "$PROJECT_NAME" == "registry-viewer" ]; then echo "Building project \"${PROJECT_NAME}\"."; \
Expand All @@ -43,7 +49,7 @@ WORKDIR /app
# Install dependencies
COPY package.json yarn.lock* ./
RUN \
if [ -f yarn.lock ]; then yarn --frozen-lockfile; \
if [ -f yarn.lock ]; then yarn --frozen-lockfile --network-timeout $YARN_TIMEOUT; \
else echo "Lockfile not found." && exit 1; \
fi

Expand Down
12 changes: 10 additions & 2 deletions apps/landing-page/project.json
Original file line number Diff line number Diff line change
Expand Up @@ -59,10 +59,18 @@
"command": "cd ./apps/landing-page/ && next-sitemap --config ./next-sitemap.config.mjs && cp -a ./dist/public/. ./dist/exported"
}
},
"docker-build": {
"docker-build-amd64": {
"executor": "nx:run-commands",
"description": "Build docker image for the linux/amd64 architecture",
"options": {
"command": "docker build -t nextjs-docker . --build-arg PROJECT_NAME=landing-page"
"command": "docker build --platform=linux/amd64 -t nextjs-docker-amd64 . --build-arg PROJECT_NAME=landing-page"
}
},
"docker-build-arm64": {
"executor": "nx:run-commands",
"description": "Build docker image for the linux/arm64 architecture",
"options": {
"command": "docker build --platform=linux/arm64 -t nextjs-docker-arm64 . --build-arg PROJECT_NAME=landing-page"
}
},
"test": {
Expand Down
12 changes: 10 additions & 2 deletions apps/registry-viewer/project.json
Original file line number Diff line number Diff line change
Expand Up @@ -58,10 +58,18 @@
"command": "cd ./apps/registry-viewer/ && next-sitemap --config ./next-sitemap.config.mjs && cp -a ./dist/public/. ./dist/exported"
}
},
"docker-build": {
"docker-build-amd64": {
"executor": "nx:run-commands",
"description": "Build docker image for the linux/amd64 architecture",
"options": {
"command": "docker build -t nextjs-docker . --build-arg PROJECT_NAME=registry-viewer"
"command": "docker build --platform=linux/amd64 -t nextjs-docker-amd64 . --build-arg PROJECT_NAME=registry-viewer"
}
},
"docker-build-arm64": {
"executor": "nx:run-commands",
"description": "Build docker image for the linux/arm64 architecture",
"options": {
"command": "docker build --platform=linux/arm64 -t nextjs-docker-arm64 . --build-arg PROJECT_NAME=registry-viewer"
}
},
"test": {
Expand Down
59 changes: 59 additions & 0 deletions scripts/build_multi_arch.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
#!/bin/sh

#
# Copyright Red Hat
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

ABSOLUTE_PATH="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
BUILD_DIR=$ABSOLUTE_PATH/..
# Due to command differences between podman and docker we need to separate the process
# for creating and adding images to a multi-arch manifest
podman=${USE_PODMAN:-false}
# Base Repository
BASE_REPO="quay.io/devfile/registry-viewer"
BASE_TAG="next"
DEFAULT_IMG="$BASE_REPO:$BASE_TAG"
# Platforms to build for
PLATFORMS="linux/amd64,linux/arm64"

if [ ${podman} == true ]; then
echo "Executing with podman"

podman manifest create "$DEFAULT_IMG"

podman build --platform="$PLATFORMS" --manifest "$DEFAULT_IMG" "$BUILD_DIR" \
--no-cache \
--build-arg PROJECT_NAME=registry-viewer \
--build-arg NEXT_PUBLIC_BASE_PATH=${NEXT_PUBLIC_BASE_PATH:-"/viewer"}

podman manifest push "$DEFAULT_IMG"

podman manifest rm "$DEFAULT_IMG"

else
echo "Executing with docker"

docker buildx create --name registry-viewer-builder

docker buildx use registry-viewer-builder

docker buildx build --push --platform="$PLATFORMS" --tag "$DEFAULT_IMG" "$BUILD_DIR" \
--no-cache \
--provenance=false \
--build-arg PROJECT_NAME=registry-viewer \
--build-arg NEXT_PUBLIC_BASE_PATH=${NEXT_PUBLIC_BASE_PATH:-"/viewer"}

docker buildx rm registry-viewer-builder

fi
Loading