-
Notifications
You must be signed in to change notification settings - Fork 2.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[CI:DOCS] Add github-action workflow to build/push multi-arch #10107
Merged
openshift-merge-robot
merged 1 commit into
containers:master
from
cevich:multi_arch_images
Apr 23, 2021
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,181 @@ | ||
| name: build multi-arch images | ||
|
|
||
| on: | ||
| # Upstream podman tends to be very active, with many merges per day. | ||
| # Only run this daily via cron schedule, or manually, not by branch push. | ||
| schedule: | ||
| - cron: '0 8 * * *' | ||
| # allows to run this workflow manually from the Actions tab | ||
| workflow_dispatch: | ||
|
|
||
| jobs: | ||
| multi: | ||
| name: multi-arch Podman build | ||
| env: | ||
| PODMAN_QUAY_REGISTRY: quay.io/podman | ||
| CONTAINERS_QUAY_REGISTRY: quay.io/containers | ||
| # list of architectures for build | ||
| PLATFORMS: linux/amd64,linux/s390x,linux/ppc64le,linux/arm64 | ||
|
|
||
| # build several images (upstream, testing, stable) in parallel | ||
| strategy: | ||
| matrix: | ||
| # Builds are located under contrib/podmanimage/<source> directory | ||
| source: | ||
| - upstream | ||
| - testing | ||
| - stable | ||
| runs-on: ubuntu-latest | ||
| # internal registry caches build for inspection before push | ||
| services: | ||
| registry: | ||
| image: quay.io/libpod/registry:2 | ||
| ports: | ||
| - 5000:5000 | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@v2 | ||
|
|
||
| - name: Set up QEMU | ||
| uses: docker/setup-qemu-action@v1 | ||
|
|
||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@v1 | ||
| with: | ||
| driver-opts: network=host | ||
| install: true | ||
|
|
||
| - name: Build and locally push Podman | ||
| uses: docker/build-push-action@v2 | ||
| with: | ||
| context: contrib/podmanimage/${{ matrix.source }} | ||
| file: ./contrib/podmanimage/${{ matrix.source }}/Dockerfile | ||
| platforms: ${{ env.PLATFORMS }} | ||
| push: true | ||
| tags: localhost:5000/podman/${{ matrix.source }} | ||
|
|
||
| # Simple verification that container works + grab version number | ||
| - name: amd64 container sniff test | ||
| id: sniff_test | ||
| run: | | ||
| VERSION_OUTPUT="$(docker run localhost:5000/podman/${{ matrix.source }} \ | ||
| podman --storage-driver=vfs version)" | ||
| echo "$VERSION_OUTPUT" | ||
| VERSION=$(grep -Em1 '^Version: ' <<<"$VERSION_OUTPUT" | awk '{print $2}') | ||
| test -n "$VERSION" | ||
| echo "::set-output name=version::${VERSION}" | ||
|
|
||
| # Generate image FQINs, labels, check whether to push | ||
| - name: Generate image information | ||
| id: image_info | ||
| run: | | ||
| if [[ "${{ matrix.source }}" == 'stable' ]]; then | ||
| # quay.io/podman/stable:vX.X.X | ||
| ALLTAGS=$(skopeo list-tags \ | ||
| docker://${{ env.PODMAN_QUAY_REGISTRY }}/stable | \ | ||
| jq -r '.Tags[]') | ||
| PUSH="false" | ||
| if fgrep -qx "$VERSION" <<<"$ALLTAGS"; then | ||
| PUSH="true" | ||
| fi | ||
|
|
||
| FQIN='${{ env.PODMAN_QUAY_REGISTRY }}/stable:v${{ steps.sniff_test.outputs.version }}' # workaround vim syntax-hilighting bug: ' | ||
| # Only push if version tag does not exist | ||
| if [[ "$PUSH" == "true" ]]; then | ||
| echo "Will push $FQIN" | ||
| echo "::set-output name=podman_push::${PUSH}" | ||
| echo "::set-output name=podman_fqin::${FQIN}" | ||
| fi | ||
|
|
||
| # quay.io/containers/podman:vX.X.X | ||
| unset ALLTAGS | ||
| ALLTAGS=$(skopeo list-tags \ | ||
| docker://${{ env.CONTAINERS_QUAY_REGISTRY }}/podman | \ | ||
| jq -r '.Tags[]') | ||
| PUSH="false" | ||
| if fgrep -qx "$VERSION" <<<"$ALLTAGS"; then | ||
| PUSH="true" | ||
| fi | ||
|
|
||
| FQIN='${{ env.CONTAINERS_QUAY_REGISTRY}}/podman:v${{ steps.sniff_test.outputs.version }}' # workaround vim syntax-hilighting bug: ' | ||
| # Only push if version tag does not exist | ||
| if [[ "$PUSH" == "true" ]]; then | ||
| echo "Will push $FQIN" | ||
| echo "::set-output name=containers_push::${PUSH}" | ||
| echo "::set-output name=containers_fqin::$FQIN" | ||
| fi | ||
| else # upstream and testing podman image | ||
| P_FQIN='${{ env.PODMAN_QUAY_REGISTRY }}/${{ matrix.source }}:master' # workaround vim syntax-hilighting bug: ' | ||
| C_FQIN='${{ env.CONTAINERS_QUAY_REGISTRY}}/podman:master' # workaround vim syntax-hilighting bug: ' | ||
| echo "Will push $P_FQIN and $C_FQIN" | ||
| echo "::set-output name=podman_fqin::${P_FQIN}" | ||
| echo "::set-output name=containers_fqin::${C_FQIN}" | ||
| # Always push 'master' tag | ||
| echo '::set-output name=podman_push::true' | ||
| echo '::set-output name=containers_push::true' | ||
| fi | ||
|
|
||
| # Hack to set $LABELS env. var. in _future_ steps. | ||
| # https://docs.github.com/en/actions/reference/workflow-commands-for-github-actions#multiline-strings | ||
| cat << EOF | tee $GITHUB_ENV | ||
| LABELS<<DELIMITER | ||
| org.opencontainers.image.source=https://github.com/${{ github.repository }}.git | ||
| org.opencontainers.image.revision=${{ github.sha }} | ||
| org.opencontainers.image.created=$(date -u --iso-8601=seconds) | ||
| DELIMITER | ||
| EOF | ||
|
|
||
| # Separate steps to login and push for podman and containers quay | ||
| # repositories are required, because 2 sets of credentials are used and `docker | ||
| # login` as well as `podman login` do not support having 2 different | ||
| # credential sets for 1 registry. | ||
| # At the same time reuse of non-shell steps is not supported by Github Actions | ||
| # via anchors or composite actions | ||
|
|
||
| # Push to 'podman' Quay repo for stable, testing. and upstream | ||
| - name: Login to 'podman' Quay registry | ||
| uses: docker/login-action@v1 | ||
| if: ${{ steps.image_info.outputs.podman_push == 'true' }} | ||
| with: | ||
| registry: ${{ env.PODMAN_QUAY_REGISTRY }} | ||
| # N/B: Secrets are not passed to workflows that are triggered | ||
| # by a pull request from a fork | ||
| username: ${{ secrets.PODMAN_QUAY_USERNAME }} | ||
| password: ${{ secrets.PODMAN_QUAY_PASSWORD }} | ||
|
|
||
| - name: Push images to 'podman' Quay | ||
| uses: docker/build-push-action@v2 | ||
| if: ${{ steps.image_info.outputs.podman_push == 'true' }} | ||
| with: | ||
| cache-from: type=registry,ref=localhost:5000/podman/${{ matrix.source }} | ||
| cache-to: type=inline | ||
| context: contrib/podmanimage/${{ matrix.source }} | ||
| file: ./contrib/podmanimage/${{ matrix.source }}/Dockerfile | ||
| platforms: ${{ env.PLATFORMS }} | ||
| push: true | ||
| tags: ${{ steps.image_info.outputs.podman_fqin }} | ||
| labels: | | ||
| ${{ env.LABELS }} | ||
|
|
||
| # Push to 'containers' Quay repo only stable podman | ||
| - name: Login to 'containers' Quay registry | ||
| if: ${{ steps.image_info.outputs.containers_push == 'true' }} | ||
| uses: docker/login-action@v1 | ||
| with: | ||
| registry: ${{ env.CONTAINERS_QUAY_REGISTRY}} | ||
| username: ${{ secrets.CONTAINERS_QUAY_USERNAME }} | ||
| password: ${{ secrets.CONTAINERS_QUAY_PASSWORD }} | ||
|
|
||
| - name: Push images to 'containers' Quay | ||
| if: ${{ steps.image_info.outputs.containers_push == 'true' }} | ||
| uses: docker/build-push-action@v2 | ||
| with: | ||
| cache-from: type=registry,ref=localhost:5000/podman/${{ matrix.source }} | ||
| cache-to: type=inline | ||
| context: contrib/podmanimage/${{ matrix.source }} | ||
| file: ./contrib/podmanimage/${{ matrix.source }}/Dockerfile | ||
| platforms: ${{ env.PLATFORMS }} | ||
| push: true | ||
| tags: ${{ steps.image_info.outputs.containers_fqin }} | ||
| labels: | | ||
| ${{ env.LABELS }} | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cat-without-args tends to raise eyebrows. Is the intention here to display those values to stdout, perhaps for an output log?
Also, I had to look up
$GITHUB_ENV, so I'm not entirely sure what it is or how it works, but shouldn't this betee -ato prevent an existing file from getting clobbered?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It should raise eyebrows...(unfortunately) like many other aspects, this construct is idiomatic to github-actions☹️ That's why I dropped the docs URL comment, and I'm glad that found some use already 😃
The file never exists, the step is suppose to create it to that triggers parsing of the contents. The
echo '::set-output name=foo::bar'idiom only works for a simple single-line value (because reasons). There's no other way to pass a multi-line value to subsequent actions (i.e.docker/build-push-action@v2). This is the best M$ GitHub could come up with 😖In buildah, the matching step uses a simple here-document redirect to a file. As you guessed, that means no record of the value shows in the logs. This is/was hiding an actual bug. I only discovered it by manually doing a
skopeo inspecton one of the images 😠 So I changed it here to useteeand make it show up in the logs to prevent future headaches.TBH I'm not sure how to make this section any clearer, it's kind of broken by the workflow "language" design 😢
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here's an example of the log output from this section as-is.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've tried twice to understand that, and failed. But it's not important, I don't really need to understand it. My point is: all the github examples use
>>(append). In the line above, you usetee(clobber), nottee -a(append). This is inconsistent with the examples documented in github.com itself, and could potentially overwrite something important, so I wanted to make sure that your use is deliberate.