Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added docker buildx support in cico scripts. #109

Closed
wants to merge 5 commits into from
Closed

Added docker buildx support in cico scripts. #109

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
b970b3b
Added buildx support in cico scripts.
prabhav-thali Jun 1, 2020
cf3297a
Used version() to validate docker version
prabhav-thali Jun 5, 2020
f868cfc
sort is used for version check
prabhav-thali Jun 25, 2020
da4897c
Using tagged image for qemu
prabhav-thali Jun 25, 2020
4c02956
Added docs for function
prabhav-thali Jul 15, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions cico_build_ci.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,5 @@ export SCRIPT_DIR

load_jenkins_vars
install_deps
check_buildx_support
build_and_push
1 change: 1 addition & 0 deletions cico_build_nightly.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,5 +22,6 @@ export SCRIPT_DIR

load_jenkins_vars
install_deps
check_buildx_support
set_nightly_tag
build_and_push
1 change: 1 addition & 0 deletions cico_build_release.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,5 +22,6 @@ export SCRIPT_DIR

load_jenkins_vars
install_deps
check_buildx_support
set_release_tag
build_and_push
53 changes: 45 additions & 8 deletions cico_functions.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,32 @@ function load_jenkins_vars() {
fi
}

# Sorts two versions and returns the lower version to check if its supported for using docker buildx.
function check_supported_version() {
local query=$1
local target=$2
echo "$target" "$query" | tr ' ' '\n' | sort -V | head -n1 2> /dev/null
}

# Checks whether docker and kernel versions are greater than or equal to the specified version, such that docker buildx requirements are met.
function check_buildx_support() {
prabhav-thali marked this conversation as resolved.
Show resolved Hide resolved
docker_version="$(docker --version | cut -d' ' -f3 | tr -cd '0-9.')"
if [[ $(check_supported_version "$docker_version" "19.03") != 19.03 ]]; then
echo "CICO: Docker $docker_version greater than or equal to 19.03 is required."
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just suspect that centos CI has the version lower than 19.03 - we might need to install new version of docker manually

exit 1
else
# Kernel
kernel_version="$(uname -r)"
if [[ $(check_supported_version "$kernel_version" "4.8") != "4.8" ]]; then
echo "Kernel $kernel_version too old - need >= 4.8." \
" Install a newer kernel."
exit 1
else
echo "kernel $kernel_version has binfmt_misc fix-binary (F) support."
fi
fi
}

function install_deps() {
# We need to disable selinux for now, XXX
/usr/sbin/setenforce 0 || true
Expand All @@ -44,6 +70,15 @@ function install_deps() {
git

service docker start

#set buildx env
export DOCKER_BUILD_KIT=1
export DOCKER_CLI_EXPERIMENTAL=enabled
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit iffy about relying on experimental features from Docker but if this is the One True Way to do multiarch builds, then so be it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The buildah discussion appears to be here: containers/buildah#1590


#Enable qemu and binfmt support
docker run --rm --privileged docker/binfmt:a7996909642ee92942dcd6cff44b9b95f08dad64
docker run --rm --privileged multiarch/qemu-user-static:4.2.0-7 --reset -p yes

echo 'CICO: Dependencies installed'
}

Expand All @@ -66,10 +101,9 @@ function set_git_commit_tag() {
export GIT_COMMIT_TAG
}

function tag_push() {
function build_and_push_using_buildx() {
local TARGET=$1
docker tag "${IMAGE}" "$TARGET"
docker push "$TARGET" | cat
docker buildx build --platform linux/amd64,linux/s390x -t "${REGISTRY}/${ORGANIZATION}/${IMAGE}:${TARGET}" -f ./build/dockerfiles/${DOCKERFILE} --push --progress plain --no-cache .
}

function build_and_push() {
Expand All @@ -86,13 +120,16 @@ function build_and_push() {

# Let's build and push image to 'quay.io' using git commit hash as tag first
set_git_commit_tag
docker build -t ${IMAGE} -f ./build/dockerfiles/${DOCKERFILE} . | cat
tag_push "${REGISTRY}/${ORGANIZATION}/${IMAGE}:${GIT_COMMIT_TAG}"
echo "CICO: '${GIT_COMMIT_TAG}' version of images pushed to '${REGISTRY}/${ORGANIZATION}' organization"

# If additional tag is set (e.g. "nightly"), let's tag the image accordingly and also push to 'quay.io'
# Create a new builder instance using buildx
docker buildx create --use --name builder
docker buildx inspect --bootstrap
build_and_push_using_buildx "${GIT_COMMIT_TAG}"
echo "CICO: '${GIT_COMMIT_TAG}' version of images pushed to '${REGISTRY}/${ORGANIZATION}' organization"

# If additional tag is set (e.g. "nightly"), let's build the image accordingly and also push to 'quay.io'
if [ -n "${TAG}" ]; then
tag_push "${REGISTRY}/${ORGANIZATION}/${IMAGE}:${TAG}"
build_and_push_using_buildx "${TAG}"
echo "CICO: '${TAG}' version of images pushed to '${REGISTRY}/${ORGANIZATION}' organization"
fi
}