Skip to content

Commit

Permalink
Human-readable incremental image suffixes
Browse files Browse the repository at this point in the history
Historically, the ID used to identify a set of images was taken from the
Cirrus-CI build ID.  This was done to make auditing easier, since one
can easily retrieve the build logs using the ID.  However, there are
many disadvantages to using the build id:

* It's not human-readable, making it difficult to ascertain exactly when
  the images were built.
* It's not guaranteed to be incremental, and therefore cannot be
  utilized as a "version".
* It doesn't convey helpful information, like which release of Fedora
  is included in the set.

For these and other reasons, switch to a simple date-based image suffix
which also includes the versions of Fedora, Prior-Fedora, and Ubuntu.

Signed-off-by: Chris Evich <[email protected]>
  • Loading branch information
cevich committed Jan 17, 2023
1 parent de5d66b commit a1e2aa6
Show file tree
Hide file tree
Showing 5 changed files with 17 additions and 20 deletions.
2 changes: 0 additions & 2 deletions .cirrus.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,6 @@ env:
CIRRUS_CLONE_DEPTH: 50
# Version of packer to use when building images
PACKER_VERSION: &PACKER_VERSION "1.8.3"
# Unique suffix label to use for all images produced by _this_ run (build)
IMG_SFX: "${CIRRUS_BUILD_ID}"

gcp_credentials: ENCRYPTED[823fdbc2fee3c27fa054ba1e9cfca084829b5e71572f1703a28e0746b1a924ee5860193f931adce197d40bf89e7027fe]

Expand Down
6 changes: 5 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ override _HLPFMT = "%-20s %s\n"

# Suffix used to identify images produce by _this_ execution
# N/B: There are length/character limitations in GCE for image names
IMG_SFX ?=
IMG_SFX := $(shell date +%y%m%d)f$(FEDORA_RELEASE)p$(PRIOR_FEDORA_RELEASE)u$(subst .,,$(UBUNTU_RELEASE))

# Env. vars needed by packer
export CHECKPOINT_DISABLE = 1 # Disable hashicorp phone-home
Expand All @@ -115,6 +115,10 @@ help: ## Default target, parses special in-line comments as documentation.
@grep -E '^[[:print:]]+:.*?## .*$$' $(MAKEFILE_LIST) | \
awk 'BEGIN {FS = ":(.*)?## "}; {printf $(_HLPFMT), $$1, $$2}'

.PHONY: img_sfx
img_sfx:
@echo "Image-Suffix: $(IMG_SFX)"

.PHONY: ci_debug
ci_debug: $(_TEMPDIR)/ci_debug.tar ## Build and enter container for local development/debugging of container-based Cirrus-CI tasks
/usr/bin/podman run -it --rm \
Expand Down
23 changes: 9 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,18 +173,18 @@ see step 4 below.

### Looking up an image ID: ###

A GCE *image ID* is simply big number prefixed by the letter 'c'. You may
A GCE *image ID* is simply the *image ID* prefixed by the letter 'c'. You may
need to look it up in a PR for example, if
[the automated comment posting github-action](.github/workflows/pr_image_id.yml)
fails. For AWS EC2 images, you'll need to look up the AMI ID (string) for each
cache-image produced.

1. In a PR, find and click the build task for the image you're interested in.
1. In a PR, find and click the *details* link for one of the build tasks.
Near the top of the Cirrus-CI WebUI, will be a section labeled 'Artifacts'.

2. Click the `manifest` artifact

3. Click the `cache_images` folder
3. Click the artifacts folder name.

4. Click the `manifest.json` file, it should open in your browser window.

Expand All @@ -195,7 +195,8 @@ cache-image produced.
a region prefix, like `us-east-1`, ignore this. At the end of the value
will be the AMI ID, similar to `ami-<big number>`. This is the ID for
this one, specific image. **Every AWS image will have a unique AMI ID**
(unlike the shared ID for GCE images).
(unlike the shared ID for GCE images). To find the common *image ID*
shared with the set, click the 'tags' tab.


## The image-builder image (overview step 1)
Expand Down Expand Up @@ -404,26 +405,20 @@ credential files and ensure correct account configuration. Having these files
stored *in your home directory* on your laptop/workstation, the process of
producing images proceeds as follows:

1. Invent some unique identity suffix for your images. It may contain (***only***)
lowercase letters, numbers and dashes; nothing else. Some suggestions
of useful values would be your name and today's date. If you manage to screw
this up somehow, stern errors will be presented without causing any real harm.

2. Ensure you have podman installed, and lots of available network and CPU
1. Ensure you have podman installed, and lots of available network and CPU
resources (i.e. turn off YouTube, shut down background VMs and other hungry
tasks). Build the image-builder container image, by executing
```
make image_builder_debug GAC_FILEPATH=</home/path/to/gac.json> \
AWS_SHARED_CREDENTIALS_FILE=</path/to/credentials> \
IMG_SFX=<ID chosen in step 1>
AWS_SHARED_CREDENTIALS_FILE=</path/to/credentials>
```

3. You will be dropped into a debugging container, inside a volume-mount of
2. You will be dropped into a debugging container, inside a volume-mount of
the repository root. This container is practically identical to the VM
produced and used in *overview step 1*. If changes are made, the container
image should be re-built to reflect them.

4. If you wish to build only a subset of available images, list the names
3. If you wish to build only a subset of available images, list the names
you want as comma-separated values of the `PACKER_BUILDS` variable. Be
sure you *export* this variable so that `make` has access to it. For
example, `export PACKER_BUILDS=ubuntu,prior-fedora`.
Expand Down
5 changes: 2 additions & 3 deletions ci/make.sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@ source "$REPO_DIRPATH/lib.sh"
# shellcheck disable=SC2154
if [[ -z "$CI" ]] || [[ "$CI" != "true" ]] || [[ "$CIRRUS_CI" != "$CI" ]]; then
die "Unexpected \$CI=$CI and/or \$CIRRUS_CI=$CIRRUS_CI"
elif [[ -z "$IMG_SFX" ]] || [[ -z "$PACKER_BUILDS" ]]; then
die "Required non-empty values for \$IMG_SFX=$IMG_SFX and \$PACKER_BUILDS=$PACKER_BUILDS"
elif [[ -z "$PACKER_BUILDS" ]]; then
die "Required non-empty values for \$PACKER_BUILDS=$PACKER_BUILDS"
elif [[ -z "$1" ]]; then
die "Build stage name is required as the first argument"
fi
Expand All @@ -37,6 +37,5 @@ fi

set -x
cd "$REPO_DIRPATH"
export IMG_SFX=$IMG_SFX
export PACKER_BUILDS=$PACKER_BUILDS
make ${1}
1 change: 1 addition & 0 deletions ci/make_container_images.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ set -eo pipefail
SCRIPT_FILEPATH=$(realpath "${BASH_SOURCE[0]}")
SCRIPT_DIRPATH=$(dirname "$SCRIPT_FILEPATH")
REPO_DIRPATH=$(realpath "$SCRIPT_DIRPATH/../")
IMG_SFX=$(make -C $REPO_DIRPATH img_sfx | grep "Image-Suffix:" | awk '{print $2}')

# shellcheck source=./lib.sh
source "$REPO_DIRPATH/lib.sh"
Expand Down

0 comments on commit a1e2aa6

Please sign in to comment.