Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support VanillaOS images #206

Draft
wants to merge 14 commits into
base: main
Choose a base branch
from
Draft

feat: support VanillaOS images #206

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
a04c3e9
refactor: create separation between ostree / vanilla / generic templates
xynydev Jul 25, 2024
019ab77
fix: circumvent vanilla not having os versions by providing a default…
xynydev Jul 25, 2024
22e599d
fix(vanilla fsguard): download genfilelist.py from vib-fsguard module
xynydev Jul 25, 2024
bf8c41f
fix: fsguard integrity check for whole /usr
xynydev Jul 25, 2024
0c02923
Merge branch 'main' into vanilla
gmpinder Jul 26, 2024
60169a5
chore: try moving & disabling some of the 'cruft'
xynydev Jul 30, 2024
e65af40
chore: try generating file list for only /usr/bin just like was done …
xynydev Jul 30, 2024
2e4ecf8
fix(fsguard): fetch FsGuard binary
xynydev Aug 1, 2024
3dcefae
fix(fsguard): correct URL
xynydev Aug 7, 2024
4d55fc0
Merge branch 'main' into vanilla
xynydev Aug 7, 2024
622f895
refactor: create new template struct for vanilla while keeping old te…
xynydev Aug 7, 2024
90c14b6
feat: allow specifying template to use in recipe with base-image-type…
xynydev Aug 7, 2024
772137d
style(generate.rs): format
xynydev Aug 7, 2024
077ace9
fix: move LABELs back to the bottom
xynydev Aug 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 2 additions & 7 deletions src/drivers.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ use std::{
};

use blue_build_recipe::Recipe;
use blue_build_utils::constants::IMAGE_VERSION_LABEL;
// use blue_build_utils::constants::IMAGE_VERSION_LABEL;
use log::{debug, info, trace};
use miette::{bail, miette, Result};
use once_cell::sync::Lazy;
Expand Down Expand Up @@ -389,12 +389,7 @@ impl Driver<'_> {
.build();
let inspection = INSPECT_DRIVER.get_metadata(&inspect_opts)?;

let os_version = inspection.get_version().ok_or_else(|| {
miette!(
help = format!("Please check with the image author about using '{IMAGE_VERSION_LABEL}' to report the os version."),
"Unable to get the OS version from the labels"
)
})?;
let os_version = inspection.get_version().unwrap_or(0);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's leave this function alone and instead call a different function for whatever the vanilla template needs. We want to try to keep the fedora templates and the vanilla templates separate as possible.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. I just did it like this because I couldn't figure out any easier way to fix this error. We should have some way to detect and declare what the base image is, and call different functions based on that, and remove the hard requirement for os_version in the tagging system, etc.

trace!("os_version: {os_version}");

os_version
Expand Down
6 changes: 5 additions & 1 deletion template/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,11 @@ use uuid::Uuid;
pub use askama::Template;

#[derive(Debug, Clone, Template, TypedBuilder)]
#[template(path = "Containerfile.j2", escape = "none", whitespace = "minimize")]
#[template(
path = "Containerfile.vanilla.j2",
escape = "none",
whitespace = "minimize"
)]
pub struct ContainerFileTemplate<'a> {
xynydev marked this conversation as resolved.
Show resolved Hide resolved
recipe: &'a Recipe<'a>,

Expand Down
0 template/templates/Containerfile.j2 → template/templates/Containerfile.ostree.j2
View file
Open in desktop
File renamed without changes.
57 changes: 57 additions & 0 deletions template/templates/Containerfile.vanilla.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
{%- import "modules/modules.j2" as modules -%}
{%- include "stages.j2" %}

# Main image
FROM {{ recipe.base_image }}:{{ recipe.image_version }} AS {{ recipe.name|replace('/', "-") }}

ARG RECIPE={{ recipe_path.display() }}
ARG IMAGE_REGISTRY={{ registry }}

{%- if self::files_dir_exists() %}
ARG CONFIG_DIRECTORY="/tmp/files"
{%- else if self::config_dir_exists() %}
ARG CONFIG_DIRECTORY="/tmp/config"
{%- endif %}
ARG MODULE_DIRECTORY="/tmp/modules"
ARG IMAGE_NAME="{{ recipe.name }}"
ARG BASE_IMAGE="{{ recipe.base_image }}"

# Key RUN
RUN --mount=type=bind,from=stage-keys,src=/keys,dst=/tmp/keys \
mkdir -p /usr/etc/pki/containers/ \
&& cp /tmp/keys/* /usr/etc/pki/containers/

# Bin RUN
RUN --mount=type=bind,from=stage-bins,src=/bins,dst=/tmp/bins \
mkdir -p /usr/bin/ \
&& cp /tmp/bins/* /usr/bin/

# Init step copied from VanillaOS template
RUN lpkg --unlock && apt-get update

{% call modules::generic_modules_run(recipe.modules_ext, os_version) %}

# Cleanup step copied from VanillaOS template
RUN apt-get autoremove -y && apt-get clean && lpkg --lock

# FsGuard step copied from VanillaOS template
# first download the required python script from the vib-fsguard module
RUN mkdir -p /sources/fsguard/ && curl https://raw.githubusercontent.com/Vanilla-OS/vib-fsguard/main/genfilelist.py -o /sources/fsguard/genfilelist.py && \
rm -rf /FsGuard && rm -f ./minisign.pub ./minisign.key && chmod +x /usr/sbin/init && mkdir /FsGuard && \
chmod +x /sources/fsguard/genfilelist.py && minisign -WG -s ./minisign.key && \
python3 /sources/fsguard/genfilelist.py /usr/bin /FsGuard/filelist /usr/sbin/FsGuard && \
minisign -Sm /FsGuard/filelist -p .//minisign.pub -s .//minisign.key && touch /FsGuard/signature && \
echo -n "----begin attach----" >> /FsGuard/signature && cat /FsGuard/filelist.minisig >> /FsGuard/signature && \
echo -n "----begin second attach----" >> /FsGuard/signature && tail -n1 .//minisign.pub >> /FsGuard/signature && \
cat /FsGuard/signature >> /sources/fsguard/FsGuard && mv /sources/fsguard/FsGuard /usr/sbin/FsGuard && rm ./minisign.key ./minisign.pub

RUN rm -fr /tmp/* /var/tmp/* /sources/*

# Labels are added last since they cause cache misses with buildah
LABEL {{ blue_build_utils::constants::BUILD_ID_LABEL }}="{{ build_id }}"
LABEL org.opencontainers.image.title="{{ recipe.name }}"
LABEL org.opencontainers.image.description="{{ recipe.description }}"
{%- if let Some(repo) = self::get_repo_url() %}
LABEL org.opencontainers.image.source="{{ repo }}"
{%- endif %}
LABEL io.artifacthub.package.readme-url=https://raw.githubusercontent.com/blue-build/cli/main/README.md
24 changes: 13 additions & 11 deletions template/templates/modules/modules.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
{% macro main_modules_run(modules_ext, os_version) %}
{% macro generic_modules_run(modules_ext, os_version) %}
# Module RUNs
{%- for module in modules_ext.modules %}
{%- if let Some(module) = module.required_fields %}

{%- if module.no_cache %}
ARG CACHEBUST="{{ build_id }}"
{%- endif %}
Expand All @@ -22,22 +23,18 @@ RUN \
{%- else %}
--mount=type=bind,from=stage-modules,src=/modules,dst=/tmp/modules,rw \
{%- endif %}
{%- if module.module_type == "akmods" %}
--mount=type=bind,from=stage-akmods-{{ module.generate_akmods_info(os_version).stage_name }},src=/rpms,dst=/tmp/rpms,rw \
{%- endif %}
--mount=type=bind,from=ghcr.io/blue-build/cli:{{ exports_tag }}-build-scripts,src=/scripts/,dst=/tmp/scripts/ \
--mount=type=cache,dst=/var/cache/rpm-ostree,id=rpm-ostree-cache-{{ recipe.name }}-{{ recipe.image_version }},sharing=locked \
/tmp/scripts/run_module.sh '{{ module.module_type }}' '{{ module.print_module_context() }}' \
&& ostree container commit
/tmp/scripts/run_module.sh '{{ module.module_type }}' '{{ module.print_module_context() }}'
{%- endif %}
{%- endif %}
{%- endfor %}
{% endmacro %}
{% macro stage_modules_run(modules_ext, os_version) %}


{% macro ostree_modules_run(modules_ext, os_version) %}
# Module RUNs
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original module macros here should remain unchanged. I would suggest making a new module macro for the vanilla OS.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I split this to generic / ostree macros, because only rpm-ostree module calls require the things added by the ostree macro, VanillaOS, stages, etc., do not and probably never will require anything like ostree container commit. If there is some sort of integration needed like the we have with the akmods module, it would be trivial to split generic into generic and vanilla and switch the vanilla template to use the new macro.

I'm envisioning that we'd also ship a generic base image type, which would not add any OS-specific things to the Containerfile, and could thus be usable for basically any operating system that supports OCI images as a distribution mechanism (without extra work from us, but with extra work from the custom image maintainer).

{%- for module in modules_ext.modules %}
{%- if let Some(module) = module.required_fields %}

{%- if module.no_cache %}
ARG CACHEBUST="{{ build_id }}"
{%- endif %}
Expand All @@ -58,9 +55,14 @@ RUN \
{%- else %}
--mount=type=bind,from=stage-modules,src=/modules,dst=/tmp/modules,rw \
{%- endif %}
{%- if module.module_type == "akmods" %}
--mount=type=bind,from=stage-akmods-{{ module.generate_akmods_info(os_version).stage_name }},src=/rpms,dst=/tmp/rpms,rw \
{%- endif %}
--mount=type=bind,from=ghcr.io/blue-build/cli:{{ exports_tag }}-build-scripts,src=/scripts/,dst=/tmp/scripts/ \
/tmp/scripts/run_module.sh '{{ module.module_type }}' '{{ module.print_module_context() }}'
--mount=type=cache,dst=/var/cache/rpm-ostree,id=rpm-ostree-cache-{{ recipe.name }}-{{ recipe.image_version }},sharing=locked \
/tmp/scripts/run_module.sh '{{ module.module_type }}' '{{ module.print_module_context() }}' \
&& ostree container commit
{%- endif %}
{%- endif %}
{%- endfor %}
{% endmacro %}
{% endmacro %}
2 changes: 1 addition & 1 deletion template/templates/stages.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ SHELL ["bash", "-c"]
{%- endif %}
{%- endif %}

{% call modules::stage_modules_run(stage.modules_ext, os_version) %}
{% call modules::generic_modules_run(stage.modules_ext, os_version) %}
{%- endif %}
{%- endfor %}
{%- endif %}
Loading