Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: build and push workflow failing due to missing -f option buildx build command #425

Merged
merged 4 commits into from
Jun 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 10 additions & 12 deletions .github/workflows/docker-publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,16 +45,17 @@ jobs:
build-images:
strategy:
matrix:
docker-platform: ["linux/arm64", "linux/amd64"]
arch: ["arm64", "amd64"]
runs-on: ubuntu-latest-m
needs: [setup, set-short-sha]
env:
SHORT_SHA: ${{ needs.set-short-sha.outputs.short_sha }}
DOCKER_PLATFORM: linux/${{ matrix.arch }}
steps:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
driver: ${{ matrix.docker-platform == 'linux/amd64' && 'docker' || 'docker-container' }}
driver: ${{ matrix.arch == 'amd64' && 'docker' || 'docker-container' }}
- name: Checkout code
uses: actions/checkout@v4
- name: Login to Quay.io
Expand All @@ -68,15 +69,15 @@ jobs:
# Clear some space (https://github.com/actions/runner-images/issues/2840)
sudo rm -rf /usr/share/dotnet /opt/ghc /usr/local/share/boost

ARCH=$(cut -d "/" -f2 <<< ${{ matrix.docker-platform }})
DOCKER_BUILDKIT=1 docker buildx build --platform=$ARCH --load \
DOCKER_BUILDKIT=1 docker buildx build --load -f Dockerfile-${{ matrix.arch }} \
Copy link
Contributor Author

@micmarty-deepsense micmarty-deepsense Jun 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this single line fixes the issue, the rest is just a refactor

--platform=$DOCKER_PLATFORM \
Copy link
Contributor Author

@micmarty-deepsense micmarty-deepsense Jun 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

technically this should be e.g. "linux/amd64" (current change), but "amd64" (as it was) would also work

--build-arg PIP_VERSION=$PIP_VERSION \
--build-arg BUILDKIT_INLINE_CACHE=1 \
--build-arg PIPELINE_PACKAGE=${{ env.PIPELINE_FAMILY }} \
--provenance=false \
--progress plain \
--cache-from $DOCKER_BUILD_REPOSITORY:$ARCH \
-t $DOCKER_BUILD_REPOSITORY:$ARCH-$SHORT_SHA .
--cache-from $DOCKER_BUILD_REPOSITORY:${{ matrix.arch }} \
-t $DOCKER_BUILD_REPOSITORY:${{ matrix.arch }}-$SHORT_SHA .
- name: Set virtualenv cache
uses: actions/cache@v4
id: virtualenv-cache
Expand All @@ -88,20 +89,17 @@ jobs:
uses: docker/setup-qemu-action@v3
- name: Test image
run: |
ARCH=$(cut -d "/" -f2 <<< ${{ matrix.docker-platform }})
source .venv/bin/activate
if [ "${{ matrix.docker-platform }}" == "linux/arm64" ]; then
DOCKER_PLATFORM="${{ matrix.docker-platform }}" DOCKER_IMAGE="$DOCKER_BUILD_REPOSITORY:$ARCH-$SHORT_SHA" \
DOCKER_IMAGE="$DOCKER_BUILD_REPOSITORY:${{ matrix.arch }}-$SHORT_SHA" \
if [ "$DOCKER_PLATFORM" == "linux/arm64" ]; then
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DOCKER_PLATFORM env variable is already set, so no need to provide it here. If you expect being explicit, please let me know

SKIP_INFERENCE_TESTS=true make docker-test
else
DOCKER_PLATFORM="${{ matrix.docker-platform }}" DOCKER_IMAGE="$DOCKER_BUILD_REPOSITORY:$ARCH-$SHORT_SHA" \
make docker-test
fi
- name: Push image
run: |
# write to the build repository to cache for the publish-images job
ARCH=$(cut -d "/" -f2 <<< ${{ matrix.docker-platform }})
docker push $DOCKER_BUILD_REPOSITORY:$ARCH-$SHORT_SHA
docker push $DOCKER_BUILD_REPOSITORY:${{ matrix.arch }}-$SHORT_SHA
publish-images:
runs-on: ubuntu-latest-m
needs: [setup, set-short-sha, build-images]
Expand Down
4 changes: 2 additions & 2 deletions Dockerfile-amd64
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# syntax=docker/dockerfile:experimental
FROM quay.io/unstructured-io/base-images:wolfi-base@sha256:6c00a236c648ffdaf196ccbc446f5c6cc9eb4e3ab9e437178abcfac710b2b373 as base
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this image does not exist anymore, I had to update SHA

FROM quay.io/unstructured-io/base-images:wolfi-base@sha256:863fd5b87e780dacec62b97c2db2aeda7f770fcf9b045b29f53ec1ddbe607b4d as base

# NOTE(crag): NB_USER ARG for mybinder.org compat:
# https://mybinder.readthedocs.io/en/latest/tutorials/dockerfile.html
Expand Down Expand Up @@ -39,4 +39,4 @@ COPY --chown=${NB_USER}:${NB_USER} scripts/app-start.sh scripts/app-start.sh
ENTRYPOINT ["scripts/app-start.sh"]
# Expose a default port of 8000. Note: The EXPOSE instruction does not actually publish the port,
# but some tooling will inspect containers and perform work contingent on networking support declared.
EXPOSE 8000
EXPOSE 8000