Error 1062 (23000): Duplicate entry when using 'docker buildx imagetools create' in Actions

[MrMeeb]

Description

I have built an action that builds a docker container on multiple nodes before merging the images together. The approach is basically a carbon copy of the approach outlined here: https://docs.docker.com/build/ci/github-actions/multi-platform/#distribute-build-across-multiple-runners

When creating the image by merging the digests of each individual image, the action fails with this error:

{"tags":["git.domain.com/mrmeeb/example-container:69.69.76","git.domain.com/mrmeeb/example-container:latest"],"labels":{"org.opencontainers.image.created":"2024-03-03T16:00:29.349Z","org.opencontainers.image.description":"","org.opencontainers.image.licenses":"","org.opencontainers.image.revision":"573fab4bdd6d5ab3dbaa39aea99140b24bbf2bd6","org.opencontainers.image.source":"https://git.domain.com/mrmeeb/example-container","org.opencontainers.image.title":"example-container","org.opencontainers.image.url":"https://git.domain.com/mrmeeb/example-container","org.opencontainers.image.version":"69.69.76"},"annotations":["manifest:org.opencontainers.image.created=2024-03-03T16:00:29.349Z","manifest:org.opencontainers.image.description=","manifest:org.opencontainers.image.licenses=","manifest:org.opencontainers.image.revision=573fab4bdd6d5ab3dbaa39aea99140b24bbf2bd6","manifest:org.opencontainers.image.source=https://git.domain.com/mrmeeb/example-container","manifest:org.opencontainers.image.title=example-container","manifest:org.opencontainers.image.url=https://git.domain.com/mrmeeb/example-container","manifest:org.opencontainers.image.version=69.69.76"]}
-t git.domain.com/mrmeeb/example-container:69.69.76 -t git.domain.com/mrmeeb/example-container:latest git.domain.com/mrmeeb/example-container@sha256:a54e0f8cd6e76d7fac0b361c18d4d9ea733f367d6f891adaccd57b6dc76c366b git.domain.com/mrmeeb/example-container@sha256:3679da30f127e3250442da966e932e81c425b61d133d91c70030d6a89100efbc
#1 [internal] pushing git.domain.com/mrmeeb/example-container:latest
#1 0.000 pushing sha256:5825d79f54f393aa01787354b551bd788a4ece6c288c6ddb7394b2c73ad3fd66 to git.domain.com/mrmeeb/example-container:latest
#1 ERROR: failed commit on ref "index-sha256:5825d79f54f393aa01787354b551bd788a4ece6c288c6ddb7394b2c73ad3fd66": unexpected status from PUT request to https://git.domain.com/v2/mrmeeb/example-container/manifests/latest: 500 Internal Server Error
#2 [internal] pushing git.domain.com/mrmeeb/example-container:69.69.76
#2 0.000 pushing sha256:5825d79f54f393aa01787354b551bd788a4ece6c288c6ddb7394b2c73ad3fd66 to git.domain.com/mrmeeb/example-container:69.69.76
#2 DONE 0.1s
------
 > [internal] pushing git.domain.com/mrmeeb/example-container:latest:
0.000 pushing sha256:5825d79f54f393aa01787354b551bd788a4ece6c288c6ddb7394b2c73ad3fd66 to git.domain.com/mrmeeb/example-container:latest
------
ERROR: failed commit on ref "index-sha256:5825d79f54f393aa01787354b551bd788a4ece6c288c6ddb7394b2c73ad3fd66": unexpected status from PUT request to https://git.domain.com/v2/mrmeeb/example-container/manifests/latest: 500 Internal Server Error

Checking Gitea logs, I see this:

gitea  | 2024/03/03 16:00:30 ...ontainer/manifest.go:452:createManifestBlob() [E] Error inserting package blob: Error 1062 (23000): Duplicate entry 'f8292f6774aeb373fb48f8f8967bafc3' for key 'package_blob.UQE_package_blob_md5'
gitea  | 2024/03/03 16:00:30 ...ntainer/container.go:89:apiError() [E] Error 1062 (23000): Duplicate entry 'f8292f6774aeb373fb48f8f8967bafc3' for key 'package_blob.UQE_package_blob_md5'

Looking at the result, I have a tagged container under 69.69.72 (excuse the random testing version), but latest was never created.

Full action for thoroughness:

name: Build-Release-Image
on:
  push:
    tags:
      - '*'

env:
  CONTAINER_NAME: git.domain.com/mrmeeb/example-container
  TEA_VERSION: 0.9.2

jobs:

  Build-Image:
    runs-on: [ubuntu-docker-latest, "${{ matrix.platform }}"]
    strategy:
      fail-fast: false
      matrix:
        platform:
          - linux/amd64
          - linux/arm64
    steps:
      - name: Prepare
        run: |
          platform=${{ matrix.platform }}
          echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
      - name: Checkout
        uses: actions/checkout@v2
      - name: Prepare tags
        id: meta
        uses: docker/metadata-action@v5
        with:
          images: ${{ env.CONTAINER_NAME }}
          tags: |
            type=pep440,pattern={{version}}
      - name: Set up QEMU
        uses: docker/setup-qemu-action@v3
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3
      - name: Login to Gitea Container Registry
        uses: docker/login-action@v3
        with:
          registry: git.domain.com
          username: ${{ env.GITHUB_ACTOR }}
          password: ${{ secrets.GTCR_TOKEN }}
      - name: Build and push by digest
        uses: docker/build-push-action@v5
        id: build
        with:
          context: ./app
          platforms: ${{ matrix.platform }}
          labels: ${{ steps.meta.outputs.labels }}
          outputs: type=image,name=${{ env.CONTAINER_NAME }},push-by-digest=true,name-canonical=true,push=true
      - name: Export digest
        run: |
          mkdir -p /tmp/digests
          digest="${{ steps.build.outputs.digest }}"
          touch "/tmp/digests/${digest#sha256:}"
      - name: Upload digest
        uses: actions/upload-artifact@v3
        with:
          name: digests-${{ env.PLATFORM_PAIR }}
          path: /tmp/digests/*
          if-no-files-found: error
          retention-days: 1
      - name: Notify
        uses: rjstone/discord-webhook-notify@v1
        if: failure()
        with:
          severity: ${{ job.status == 'success' && 'info' || (job.status == 'cancelled' && 'warn' || 'error') }}
          details: Build ${{ job.status == 'success' && 'succeeded' || (job.status == 'cancelled' && 'cancelled' || 'failed') }}!
          webhookUrl: ${{ secrets.DISCORD_WEBHOOK }}
          username: Gitea
          avatarUrl: ${{ vars.RUNNER_ICON_URL }}

  Merge-Images:
    runs-on: ubuntu-docker-latest
    needs: [Build-Image]
    steps:
      - name: Download digests
        uses: actions/download-artifact@v3
        with:
          path: /tmp/digests
          pattern: digests-*
          merge-multiple: true
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3
      - name: Prepare Docker metadata
        id: meta
        uses: docker/metadata-action@v5
        with:
          images: ${{ env.CONTAINER_NAME }}
      - name: Login to Gitea Container Registry
        uses: docker/login-action@v3
        with:
          registry: git.domain.com
          username: ${{ env.GITHUB_ACTOR }}
          password: ${{ secrets.GTCR_TOKEN }}
      - name: Create manifest list and push
        working-directory: /tmp/digests
        run: |
          echo $DOCKER_METADATA_OUTPUT_JSON
          echo $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
            $(printf '${{ env.CONTAINER_NAME }}@sha256:%s ' $(ls -p */* | cut -d / -f 2))
          docker buildx imagetools create $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
            $(printf '${{ env.CONTAINER_NAME }}@sha256:%s ' $(ls -p */* | cut -d / -f 2))
      - name: Inspect image
        run: |
          docker buildx imagetools inspect ${{ env.CONTAINER_NAME }}:${{ steps.meta.outputs.version }}          
      - name: Notify
        uses: rjstone/discord-webhook-notify@v1
        if: failure()
        with:
          severity: ${{ job.status == 'success' && 'info' || (job.status == 'cancelled' && 'warn' || 'error') }}
          details: Build ${{ job.status == 'success' && 'succeeded' || (job.status == 'cancelled' && 'cancelled' || 'failed') }}!
          webhookUrl: ${{ secrets.DISCORD_WEBHOOK }}
          username: Gitea
          avatarUrl: ${{ vars.RUNNER_ICON_URL }}

  Create-Release:
    runs-on: [ubuntu-latest, linux/amd64]
    needs: [Merge-Images]
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Get tag
        run: echo "RELEASE_VERSION=${GITHUB_REF#refs/*/}" >> $GITHUB_ENV
      - name: Prepare tea
        run: |
          # Download tea from Gitea release page
          echo "Downloading Tea v${{ env.TEA_VERSION }}" && \
          wget -q -O tea https://gitea.com/gitea/tea/releases/download/v${{ env.TEA_VERSION }}/tea-${{ env.TEA_VERSION }}-linux-amd64 && \
          echo "Downloaded Tea" && \
          chmod +x tea && \
          # Login to Gitea
          echo "Logging in to Gitea using Tea" && \
          ./tea login add --name Gitea --url https://git.domain.com --token ${{ secrets.GITHUB_TOKEN }} && \
          echo "Done"
      - name: Make release
        run: |
          echo "Creating release" && \
          ./tea release create --login "Gitea" --repo ${{ env.GITHUB_REPOSITORY }} --tag ${{ env.RELEASE_VERSION }} -t ${{ env.RELEASE_VERSION }} && \
          echo "Done"
      - name: Notify
        uses: rjstone/discord-webhook-notify@v1
        if: failure()
        with:
          severity: ${{ job.status == 'success' && 'info' || (job.status == 'cancelled' && 'warn' || 'error') }}
          details: Release ${{ job.status == 'success' && 'succeeded' || (job.status == 'cancelled' && 'cancelled' || 'failed') }}!
          webhookUrl: ${{ secrets.DISCORD_WEBHOOK }}
          username: Gitea
          avatarUrl: ${{ vars.RUNNER_ICON_URL }}

  Notify:
    runs-on: ubuntu-latest
    needs: [Build-Image, Merge-Images, Create-Release]
    steps:
      - name: Notify
        uses: rjstone/discord-webhook-notify@v1
        if: always()
        with:
          severity: ${{ job.status == 'success' && 'info' || (job.status == 'cancelled' && 'warn' || 'error') }}
          details: Release ${{ job.status == 'success' && 'succeeded' || (job.status == 'cancelled' && 'cancelled' || 'failed') }}!
          webhookUrl: ${{ secrets.DISCORD_WEBHOOK }}
          username: Gitea
          avatarUrl: ${{ vars.RUNNER_ICON_URL }}

Gitea Version

1.21.6

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Screenshots

No response

Git Version

No response

Operating System

No response

How are you running Gitea?

Gitea 1.21.6 running in Docker on an Ubuntu host, with Minio storage for packages only.

Database

MySQL/MariaDB

[lunny]

This should be closed by #32022