GitHub Actions

This page contains any notes around how GitHub Actions is used to build, test, and publish the project that don't make sense to directly include in the workflow file.

Multi-architecture / Multi-platform container images

Starting in v1.6.0 multiarch images are published for both the driver and operator. This section includes any notes that may be helpful understanding how multiarch binaries/images are built and maintained for the project.

Driver Notes

We use `release-tools to build binaries and assemble chwrap tarfiles for multiple architectures:

make BUILD_PLATFORMS="linux amd64 amd64 amd64;linux arm64 arm64 arm64" all

We do not use release-tools to actually publish images using GitHub actions, mainly because it is opinionated about how images should be tagged. For images built from a PR we want to tag images with the SHA of the GitHub commit that triggered the build so we can later reuse the image for e2e testing. This is not possible without modifying release-tools, but it is generally discouraged from making updates to release-tools directly, as it makes it difficult to update our version of release-tools from the upstream repository (and there is a build check preventing us from doing so).

For development it is possible to build/push multiarch images to a custom registry using release-tools: make BUILD_PLATFORMS="linux amd64 amd64 amd64;linux arm64 arm64 arm64" PULL_BASE_REF=master REGISTRY_NAME=ghcr.io/iamjoemccormick push-multiarch

Normally though this is unnecessary and you can just build the driver container for your current platform: make container

Other notes on release-tools and multiarch images:

• release-tools will automatically append the name of the of the cmd/* directory (i.e., beegfs-csi-driver) to the REGISTRY_NAME. This wouldn't cause a problem using release-tools with GitHub Actions because you could use something like REGISTRY_NAME=ghcr.io/iamjoemccormick/test to differentiate test versus production images. Thus the lack of control over tags is still the only show stopper to directly using release-tools for publishing images.

• release-tools expects BUILD_PLATFORMS=linux amd64 amd64 amd64;linux arm64 arm64 arm64 where the third argument represents buildx_platform. Normally buildx_platform is in the format linux/amd64 but release-tools automatically prepends the os. For example you would do --platform linux/amd64,linux/arm64 if directly calling docker buildx build. This is why the workflow has to specify platforms in two ways using RELEASE_TOOLS_BUILD_PLATFORMS and DOCKER_BUILDX_BUILD_PLATFORMS.

Helpful things when debugging:

• You can inspect the manifests (to make sure all the expected architectures are present) with docker manifest inspect ghcr.io/thinkparq/test-beegfs-csi-driver:4aa9e87c9f8f3cd0eb232c334be6517c0323ed97

Helpful references:

• https://speedscale.com/blog/how-to-build-multi-arch-docker-images/
• https://www.docker.com/blog/faster-multi-platform-builds-dockerfile-cross-compilation-guide/
• On when to use $BUILDPLATFORM versus $TARGETPLATFORM
  • https://docs.docker.com/build/building/multi-platform/#cross-compilation
  • https://stackoverflow.com/questions/73978929/how-to-build-docker-image-for-multiple-platforms-with-cross-compile
  • TL;DR: $BUILDPLATFORM is used for intermediate stages that cross compile or output non-platform specific results that can be copied into the target stage which should use $TARGETPLATFORM.

Operator Notes

We have more control over how the operator is build/tested because it does not use release-tools. However to be consistent with how the driver is built, we follow the same approach to build multiarch binaries and test using Make, but use GitHub actions to actually build and push multiarch container images.

There is a Make docker-buildx target that could be updated to build multiarch images for development purposes but needs updating to work correctly and is currently disabled (and will fail if targeted).