Skip to content

Concepts of building

Jump to bottom
accetto edited this page Nov 6, 2022 · 10 revisions

Concepts of building

Version: G3v2

Updated: 2022-11-04

Introduction

This page describes the concepts that should help to understand the specifics of the used building process.

The page does not describe the common and well-known concepts of building Docker images.

The concepts described here are specific to this project and they should help by achieving the project goals.

The page should also help to understand how the actual implementation works.

It can help to read about the Concepts of Dockerfiles first.

Concepts of building process

There are actually two different building process concepts. The first one is about building a single image and the other one is about building and publishing sets of images into the Docker Hub repositories.

Concept of building a single image

Building a single image is generally a two step process.

Building helper image

In the first step a temporary helper image is built. It is used only for getting the current values of the version stickers. The concept of the version stickers is described on the page Concepts of Dockerfiles.

The current value of the verbose version sticker is then compared to the verbose version sticker of the previously published image. If there is no change, then there is no need to refresh the image and the rest of the building pipeline can be skipped.

For this concept to work, there is a need of persisting the verbose version sticker values somewhere outside the image. The current implementation of the building pipeline uses the GitHub Gists.

Strictly speaking, the building of the helper image is necessary only if the final persistent image needs to include the updated value of the short version sticker in its metadata (labels). Otherwise the first building step can be skipped completely.

On the other hand, only this step is required for checking if the image needs a refresh.

Building final image

If the value of the verbose version sticker has changed since the last release, then the final persistent image is built in the second step.

This time also its labels created and version-sticker are set to the updated values from the temporary helper image.

Concept of building sets of images

Building and publishing sets of images involves two kinds of conceptual repositories that play two different roles described below.

The reasons for this decision include the following:

  • Having more freedom for experimenting by developing new images
  • Possibility to use repositories of different visibility (private or public)
  • Docker Hub does not support renaming of repositories
  • Dockerfiles and README files published automatically by building on the Docker Hub are not always the correct ones
  • Automatic publishing of README files by building on on the Docker Hub is not flexible enough

Concept of separated image repositories

Concept of a builder repository

The first kind of the conceptual repositories is referenced as a builder repository.

There is only one builder repository and it is essentially just a repository name.

All the images built on the building stage (local or another) will get this name and they will differ only by their tags.

The builder repository is not bound to the GitHub repository containing the resources for building the images. The GitHub repository needs to be cloned to the building stage.

Strictly speaking, the current implementation of the building pipeline allows using the builder repository as a secondary deployment repository during the development, but that is not an actual part of the concept.

Concept of deployment repositories

The second kind of the conceptual repositories is referenced as a deployment repository.

The deployment repository is one or more actual repository/repositories on the Docker Hub where the final images will be published.

There is a one-to-many relation between the builder repository and the deployment repositories.

The names of the deployment repositories and the tags of the images published there are independent from the ones used by the builder repository.

The deployment repositories are not bound to the GitHub repository used by the builder repository. They also can be public or private.

They are no Dockerfiles in the deployment repositories. The links to the Dockerfiles should be contained in the README files.

The README files for the deployment repositories are prepared off-line by a provided utility and then transferred to the Docker Hub.

Concept of metadata stores

Achieving some of the project goals and supporting the concept of single image building requires independent stores for persisting the required metadata.

There should be two physically separated metadata stores - one for the builder repository and one for the deployment repositories. However, conceptually there is no difference between them and they also contain the same kinds of metadata. They also can be private or public.

Conceptually any persistent data store can be used. However, the current building pipeline implementation uses the GitHub Gists.

Currently there are the following kinds of metadata stored in the GitHub Gists:

  • JSON endpoint for the created badge, which displays the time and date when the final image has been built
  • JSON endpoint for the version-sticker badge, which displays the short version sticker value
  • content of the verbose version sticker, which is used by the building pipeline and which is also linked with the version-sticker badge in the README file

It should be understood, that the GitHub Gist belonging to the deployment repositories merely stores the metadata (e.g. badge endpoints) of the published images. It is maintained by the building pipeline, but it does not participate in the decision making if an image needs to be refreshed. Only the verbose version sticker values from the GitHub Gist belonging to the builder repository are taken into the consideration.

Other building concepts

The following concepts allow better separation of concerns by implementing the building pipeline.

Concept of building branches

The building branch is roughly the same concept as the source branch of the auto-builder on the Docker Hub.

However, the source branch on the Docker Hub means always the used git branch on the GitHub.

By building locally there is no need for such tight binding. The building branch is just a concept of a set of building scenarios.

Concept of building blends

This concept is related to the concept of features, because it means a mix of features of the particular image build.

Clone this wiki locally