Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for manifests using docker experimental cli commands #622

Merged
merged 3 commits into from
Sep 8, 2018
Merged

Support for manifests using docker experimental cli commands #622

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
8c16b0f
Add manifest support for release images
mkumatag Feb 10, 2018
0c3bd19
Add docker checks
mkumatag Aug 29, 2018
e4eef22
Use DOCKER_CLI_EXPERIMENTAL to enable docker CLI experimental features
dims Sep 5, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions anago
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -348,6 +348,30 @@ check_prerequisites () {

security_layer::auth_check 2 || return 1

logecho -n "Checking Docker version: "
docker_version=$(docker version --format '{{.Client.Version}}' | cut -d"-" -f1)
if [[ ${docker_version} != 18.06.0 && ${docker_version} < 18.06.0 ]]; then
logecho "Minimum docker version 18.06.0 is required for " \
"creating and pushing manifest images[found: ${docker_version}]"
return 1
fi
logecho -r "$OK"

# Ensure that the docker command line supports the manifest images
export DOCKER_CLI_EXPERIMENTAL=enabled

# TODO: Remove this section once docker manifest command promoted
# from Experimental
logecho -n "Verifying Docker CLI Experimental status: "
cli_experimental=$(docker version --format '{{.Client.Experimental}}' | cut -d"-" -f1)
if [[ "${cli_experimental}" == "false" ]]; then
logecho "Docker Client Experimental flag is false, should be enabled to " \
"push the manifest images"
logecho "More info: https://docs.docker.com/edge/engine/reference/commandline/manifest_create/"
return 1
fi
logecho -r "$OK"

if ! ((FLAGS_gcb)); then
ensure_gcp_users || return 1
fi
Expand Down
4 changes: 1 addition & 3 deletions build/Dockerfile.k8s-cloud-builder
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,5 @@ RUN \
stable edge" && \
apt-get -y update

# As of 2018-04-10, leaving this pinned to 17.09 due to image pull issues
# with 17.12
ARG DOCKER_VERSION=17.09.0~ce-0~ubuntu
ARG DOCKER_VERSION=18.06.0~ce~3-0~ubuntu
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

18.06.1~ce~3-0~ubuntu is already out, just see if you can use that version..

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mkumatag we kinda tested the 18.06.0 bump yesterday so let's leave it alone unless 18.06.1 is absolutely necessary

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this same pinning is in place in test-infra for docker in docker building (we talked to releng about this) because we started flaking a ton on image pull around then. we've also not upgraded.. filing a bug for us to upgrade as well

RUN apt-get install -y docker-ce=${DOCKER_VERSION} unzip
42 changes: 24 additions & 18 deletions lib/releaselib.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -904,6 +904,7 @@ release::docker::release () {
local -a new_tags
local new_tag
local binary
local -A manifest_images

if [[ "$registry" == "$GCRIO_PATH_PROD" ]]; then
# Switch to the push alias if using the $GCRIO_PATH_PROD alias
Expand All @@ -924,28 +925,33 @@ release::docker::release () {
fi
binary=${BASH_REMATCH[1]}

# If amd64, tag both the legacy tag and -amd64 tag
if [[ "$arch" == "amd64" ]]; then
# binary may or may not already contain -amd64, so strip it first
new_tags=(\
"$push_registry/${binary/-amd64/}:$version"
"$push_registry/${binary/-amd64/}-amd64:$version"
)
else
new_tags=("$push_registry/$binary:$version")
fi
new_tag="$push_registry/${binary/-$arch/}"
new_tag_with_arch=("$new_tag-$arch:$version")
manifest_images["${new_tag}"]+=" $arch"

logrun docker load -qi $tarfile
for new_tag in ${new_tags[@]}; do
logrun docker tag $orig_tag $new_tag
logecho -n "Pushing $new_tag: "
# TODO: Use docker direct when fixed later
#logrun -r 5 -s docker push "$new_tag" || return 1
logrun -r 5 -s $GCLOUD docker -- push "$new_tag" || return 1
done
logrun docker rmi $orig_tag ${new_tags[@]} || true
logrun docker tag $orig_tag ${new_tag_with_arch}
logecho -n "Pushing ${new_tag_with_arch}: "
# TODO: Use docker direct when fixed later
#logrun -r 5 -s docker push "${new_tag_with_arch}" || return 1
logrun -r 5 -s $GCLOUD docker -- push "${new_tag_with_arch}" || return 1
logrun docker rmi $orig_tag ${new_tag_with_arch} || true

done
done

for image in "${!manifest_images[@]}"; do
local archs=$(echo "${manifest_images[$image]}" | sed -e 's/^[[:space:]]*//')
local manifest=$(echo $archs | sed -e "s~[^ ]*~$image\-&:$version~g")
# This command will push a manifest list: "${registry}/${image}-ARCH:${version}" that points to each architecture depending on which platform you're pulling from
logecho "Creating manifest image ${image}:${version}..."
logrun -r 5 -s docker manifest create --amend ${image}:${version} ${manifest} || return 1
for arch in ${archs}; do
logecho "Annotating ${image}-${arch}:${version} with --arch ${arch}..."
logrun -r 5 -s docker manifest annotate --arch ${arch} ${image}:${version} ${image}-${arch}:${version} || return 1
done
logecho "Pushing manifest image ${image}:${version}..."
logrun -r 5 -s docker manifest push ${image}:${version} || return 1
done

# Always reset back to $GCP_USER
Expand Down