Skip to content

Commit

Permalink
feat: multi-arch builds, new CLI arguments env var
Browse files Browse the repository at this point in the history
  • Loading branch information
doublecompile committed Apr 19, 2024
1 parent 074d94f commit 2e179fc
Show file tree
Hide file tree
Showing 5 changed files with 159 additions and 7 deletions.
15 changes: 15 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
name: build

on:
pull_request:
- main
workflow_dispatch: {}

jobs:
shellcheck:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: ShellCheck
uses: ludeeus/[email protected]
127 changes: 127 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
name: release

on:
push:
branches:
- main

env:
REGISTRY_IMAGE: mindgrub/mysqldump-to-s3
IMAGE_VERSION: "1"

jobs:
build:
runs-on: ubuntu-latest

strategy:
fail-fast: true
matrix:
platform:
- "linux/amd64"
- "linux/arm64"
alpine_version:
- "3.19"
- "3.18"
- "3.17"

steps:
- name: Prepare
run: |
platform=${{ matrix.platform }}
echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
- name: Checkout
uses: actions/checkout@v4

- name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY_IMAGE }}

- name: Set up QEMU
uses: docker/setup-qemu-action@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Build and push by digest
id: build
uses: docker/build-push-action@v5
with:
context: .
build-args: |
ALPINE_VERSION=${{ matrix.alpine_version }}
platforms: ${{ matrix.platform }}
labels: ${{ steps.meta.outputs.labels }}
outputs: type=image,name=${{ env.REGISTRY_IMAGE }},push-by-digest=true,name-canonical=true,push=true
tags: ${{ steps.meta.outputs.tags }}

- name: Export digest
run: |
mkdir -p /tmp/digests
digest="${{ steps.build.outputs.digest }}"
touch "/tmp/digests/${digest#sha256:}"
- name: Upload digest
uses: actions/upload-artifact@v4
with:
name: digests-${{ env.PLATFORM_PAIR }}
path: /tmp/digests/*
if-no-files-found: error
retention-days: 1

manifest:
runs-on: ubuntu-latest

needs: build

strategy:
fail-fast: false
matrix:
alpine_version:
- "3.19"
- "3.18"
- "3.17"

steps:
- name: Download digests
uses: actions/download-artifact@v4
with:
path: /tmp/digests
pattern: digests-*
merge-multiple: true

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY_IMAGE }}
# type=raw,value=latest,enabled=${{ matrix.alpine_version == '3.19' }}
tags: |
type=raw,value=${{ env.IMAGE_VERSION }}-alpine-${{ matrix.alpine_version }}
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Create manifest list and push
working-directory: /tmp/digests
run: |
docker buildx imagetools create $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
$(printf '${{ env.REGISTRY_IMAGE }}@sha256:%s ' *)
- name: Inspect image
run: |
docker buildx imagetools inspect ${{ env.REGISTRY_IMAGE }}:${{ steps.meta.outputs.version }}
6 changes: 3 additions & 3 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
FROM alpine:3.18
ARG ALPINE_VERSION=3.19
FROM alpine:${ALPINE_VERSION}

RUN apk add --no-cache mysql-client python3 py3-pip coreutils jq \
&& pip install awscli
RUN apk add --no-cache mysql-client python3 py3-pip coreutils jq aws-cli

ADD run.sh /run.sh

Expand Down
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,10 @@ An Alpine-based Docker image for producing a file with `mysqldump` and uploading
- `FILENAME_PREFIX` – Optional. A string to prepend to the exported file (Default: "snapshot").
- `S3_STORAGE_TIER` – Optional. The storage tier to use with S3 (Default: "STANDARD_IA").
- `S3_PREFIX` – Optional. A string to prepend to the S3 object key (Default: "").
- `MYSQL_NET_BUFFER_LENGTH` – Optional. The `net_buffer_length` setting for `mysqldump` (Default: "16384").
- `MYSQLDUMP_OPTS` – Optional. Additional command line options to provide to `mysqldump`.
- `REQUESTOR` – Optional. The email address of the user who requested this dump to be stored in the S3 metadata.
- `SFN_TASK_TOKEN` – Optional. A Step Functions [Task Token](https://docs.aws.amazon.com/step-functions/latest/apireference/API_GetActivityTask.html#StepFunctions-GetActivityTask-response-taskToken). If present, this token will be used to call [`SendTaskHeartbeat`](https://docs.aws.amazon.com/step-functions/latest/apireference/API_SendTaskHeartbeat.html) and [`SendTaskSuccess`](https://docs.aws.amazon.com/step-functions/latest/apireference/API_SendTaskSuccess.html). The task output sent to `SendTaskSuccess` will consist of a JSON object with a single property: `uri` (containing the S3 URI of the database dump).
- `MYSQL_NET_BUFFER_LENGTH`_[Deprecated]_ Optional. The `net_buffer_length` setting for `mysqldump`.

### AWS Permissions

Expand All @@ -25,7 +26,7 @@ If this Docker image is used within Amazon ECS, specify permissions to S3 (and o

Since this image is based on Alpine, the version of `mysqldump` in this image is actually MariaDB.

The following CLI arguments are included in the call to `mysqldump`: `-R -E --triggers --single-transaction --comments`.
The following CLI arguments are included in the call to `mysqldump`: `-R -E --triggers --single-transaction --comments`. Use the `MYSQLDUMP_OPTS` environment variable to specify additional options.

The database host, database name, and requestor are all added to the S3 object metadata.

Expand Down
13 changes: 11 additions & 2 deletions run.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
#!/bin/sh
set -e
set -o pipefail

# Send heartbeat
if [ -n "$SFN_TASK_TOKEN" ]; then
Expand All @@ -8,7 +9,6 @@ fi

# Variable defaults
: "${FILENAME_PREFIX:=snapshot}"
: "${MYSQL_NET_BUFFER_LENGTH:=16384}"
: "${S3_STORAGE_TIER:=STANDARD_IA}"
: "${DB_PORT:=3306}"

Expand All @@ -19,8 +19,17 @@ destination="/data/$filename"
s3_url="s3://${S3_BUCKET}/${S3_PREFIX}${filename}"

# Export the database
set -- -h "$DB_HOST" -u "$DB_USER" --password="$DB_PASS" -P "$DB_PORT" -R -E --triggers --single-transaction --comments
if [ -n "$MYSQL_NET_BUFFER_LENGTH" ]; then
set -- "$@" --net-buffer-length="$MYSQL_NET_BUFFER_LENGTH"
fi
if [ -n "$MYSQLDUMP_OPTS" ]; then
set -- "$@" $MYSQLDUMP_OPTS
fi
set -- "$@" "$DB_NAME"
mysqldump_opts=$(printf ' %s' "$@")
echo "About to export mysql://$DB_HOST/$DB_NAME to $destination"
mysqldump -h "$DB_HOST" -u "$DB_USER" --password="$DB_PASS" -P "$DB_PORT" -R -E --triggers --single-transaction --comments --net-buffer-length="$MYSQL_NET_BUFFER_LENGTH" "$DB_NAME" | gzip > "$destination"
eval "mysqldump $mysqldump_opts" | gzip > "$destination"
echo "Export to $destination completed"

# Send heartbeat
Expand Down

0 comments on commit 2e179fc

Please sign in to comment.