Skip to content

Building stages

Jump to bottom
accetto edited this page Mar 10, 2023 · 10 revisions

Building stages

Version: G3v2

Updated: 2023-08-10

Introduction

Environments for building the images will be referred as stages.

Since the second version (G3v2) is the local stage the default building stage.

Other supported building stages (e.g. the GitLab) are conceptually not different from the local stage.

Building on the Docker Hub stage has been abandoned, because the Docker Hub has removed the auto-building feature from the free plan.

It should be noticed, that the version G3v2 of the building pipeline offers a significantly higher performance by building sets of images or by repeated builds. It's achieved by utilizing the local g3-cache, which is described separately.

Local building stage

The local building stage is any computer with Docker or Docker Desktop installed. Both Linux and Windows operating systems are supported. Mac OS has not been tested, but it's expected to work equally well.

It is recommended to use the Visual Studio Code on the local building stage, but it is not a requirement.

Building the images on the local stage offers a lot of flexibility.

For example:

  • It is possible to execute the complete building pipeline or only a part of it.

  • It is possible to build the final images and push them into the deployment repositories on the Docker Hub.

  • It is possible to build images for private local use, skipping the updating of the metadata in the GitHub Gists and the pushing to the Docker Hub.

  • It is possible to modify and debug the hook scripts, the Dockerfiles, the startup scripts and to do all the stuff the developers usually like to do.

There is no special configuration required on the local building stage. The scripts and the environment variables are the same on all building stages.

To make it more convenient, there is the file example-secrets.rc, which can be modified and sourced in the terminal when beginning the building session session.

The local building example describes how to build images on the local stage. It makes use of the provided utilities builder.sh and ci-builder.sh.

There is also the utility util-readme.sh, which is used for preparing the README files for the Docker Hub.

Environment variables

Before executing the building pipeline, some environment variables must be set.

Not all environment variables are required in each scenario.

Some of the environment variables contain secrets and they should be used carefully. The secrets should not be written into the logs, for example.

To make it more convenient, the project includes the file example-secrets.rc, which contains all the environment variables used by the building pipeline. This file can be modified and sourced in the terminal window (at shell level zero) in the beginning of each building session.

For example, if the example file has been modified, renamed to my-secrets.rc and stored into the current directory, then it can be source with the following statement:

source ./docker/hooks/my-secrets.rc

If the file is named secrets.rc and copied into the folder docker/hooks/, then it is sourced automatically by the hook script env.rc.

Non-secret variables

Variable DOCKER_BUILDKIT

This variable should be set to 1 to ensure, that the BuildKit is used for building.

Variable FORCE_BUILDING

If set to 1, then the final persistent image will be built even if its verbose version sticker value has not changed since its last release.

Variable PROHIBIT_BUILDING

If set to 1, then the image will not be built even if its verbose version sticker value has changed since its last release.

Variable VERSION_STICKER_PREFIX

Defines the string, which will be used as the prefix of the short version sticker value.

Variable VERSION_STICKER_SUFFIX

Defines the string, which will be used as the suffix of the short version sticker value.

Variable REPO_OWNER_NAME

Defines the name of the owner of the deployment repositories on the Docker Hub. For example, REPO_OWNER_NAME="accetto".

This variable is mandatory and must be always set.

Variable GIST_ID

This is the ID of the Github Gist belonging to the builder repository. It stores the metadata used by building process. The verbose version stickers from this GitHub Gist are used by deciding if a particular image needs a refresh.

Variable DEPLOY_GIST_ID

This is the ID of the GitHub Gist belonging to the deployment repositories. It contains the metadata used by the README file badges.

Variables DEPLOYMENT_REPO[_*]

The set of variables containing the names of the deployment repositories. For example, DEPLOYMENT_REPO="ubuntu-vnc-xfce-g3", DEPLOYMENT_REPO_CHROMIUM="ubuntu-vnc-xfce-chromium-g3", DEPLOYMENT_REPO_FIREFOX="ubuntu-vnc-xfce-firefox-g3".

The reserved repository name void will block publishing to the repository.

Variable BUILDER_REPO

The name of the building repository. For example, BUILDER_REPO="headless-ubuntu-g3".

This variable is mandatory and must be always set.

Variable FORCE_PUBLISHING_BUILDER_REPO

The current building pipeline implementation supports using the builder repository as the secondary deployment repository during development.

If this variable is set to 1, then the images will be pushed also to the builder repository.

The repository with the BUILDER_REPO name must already exist on the Docker Hub in that case.

Variable KEEP_HELPER_FILES

If set to 1, then the hook script post_push will not remove the temporary helper files as it normally does.

Variable SHARED_G3_CACHE_PATH

This variable contains the absolute path to the shared g3-cache, which can be used for initializing the local g3-cache, which must be always placed inside the Docker building context. The g3-cache is described separately.

Secret environment variables

Variable GIST_TOKEN

This secret allows access and manipulation of the GitHub Gists.

Its value is the GitHub PAT (personal access token) secret value, which is created in the GitHub repository containing the resources for the images (Settings/Developer settings/Personal access tokens).

The name of the PAT is not important, but it must have the gist scope permission, allowing to access the GitHub Gists.

It should be understood that this PAT could have access to all the GitHub Gists of the same GitHub owner (account).

This secret is created on the GitHub and it is used for modifying the GitHub Gists.

Variables DOCKERHUB_USERNAME, DOCKERHUB_PASSWORD

These secrets are required for pushing the images to the Docker Hub.

Their values are the Docker Hub account's name and password.

These secrets are created on the Docker Hub and they used for accessing the repositories on the Docker Hub.

Clone this wiki locally