Live-CD - Build Linux Only

[stef204]

Is it possible to build the iso only for Linux environment (as opposed to building also for Windows, etc.)?
If so, how to accomplish that?

[yveszoundi]

I'll convert this issue into a discussion item and answer in a following thread.
This will likely lead to a ticket for documentation updates.

[yveszoundi]

Please follow the notes below and let me know if you run into issues.

General notes

The ISO itself is Linux Only (based on Debian), and it includes Linux binaries in the ISO.
There are 2 ISO images (amd64 and arm64 -> mostly apple M1 users)

Live CD minimal steps

I'll assume that you want to build only for amd64/x86_64, and that you do not care about arm.
You can build only Linux binaries and then the Live CD.

• Restricts generated artifacts to amd64 only binaries
  • Change the value of CPU_ARCHS to "amd64" in ci_cd/linux/build.sh
  • Change the value of CPU_ARCHS to "amd64" In ci_cd/live_cd/build.sh
• Build Linux binaries for amd64 ./ci_cd/linux/build.sh
• Proceed with the Live CD creation ./ci_cd/live_cd/build.sh

Additional notes

Similarly to Linux, you could generate only Windows or Mac OS executables by running ci_cd/windows/build.sh or ci_cd/macos/build.sh.

[stef204]

You are correct in your assumptions. I will follow the steps above and post the result here. Thank you.

[stef204]

So, following the above steps, it fails here (looks like a permission error):

%./ci_cd/linux/build.sh
+ podman run --platform linux/amd64 --rm --privileged -v /mnt/ssdd/Downloads/src/entrusted/app:/src -v /mnt/ssdd/Downloads/src/entrusted/app/../artifacts:/artifacts -v /mnt/ssdd/Downloads/src/entrusted/app/../ci_cd:/ci_cd docker.io/uycyjnzgntrn/rust-linux:1.67.0 /bin/sh -c 'cd /src && CARGO_NET_GIT_FETCH_WITH_CLI=true CARGO_NET_RETRY=10 cargo build --target x86_64-unknown-linux-gnu --release --features=gui  --manifest-path /src/entrusted_client/Cargo.toml --bin entrusted-gui && CARGO_NET_GIT_FETCH_WITH_CLI=true CARGO_NET_RETRY=10 cargo build --release --target x86_64-unknown-linux-musl --manifest-path /src/entrusted_client/Cargo.toml --bin entrusted-cli && CARGO_NET_GIT_FETCH_WITH_CLI=true CARGO_NET_RETRY=10 RUSTFLAGS='\''-C target-feature=+crt-static'\'' cargo build --release --target x86_64-unknown-linux-musl --manifest-path /src/entrusted_webserver/Cargo.toml && CARGO_NET_GIT_FETCH_WITH_CLI=true CARGO_NET_RETRY=10 RUSTFLAGS='\''-C target-feature=+crt-static'\'' cargo build --release --target x86_64-unknown-linux-musl --manifest-path /src/entrusted_webclient/Cargo.toml ; strip /src/entrusted_webclient/target/x86_64-unknown-linux-musl/release/entrusted-webclient && strip /src/entrusted_webserver/target/x86_64-unknown-linux-musl/release/entrusted-webserver && strip /src/entrusted_client/target/x86_64-unknown-linux-musl/release/entrusted-cli; cp /src/entrusted_webclient/target/x86_64-unknown-linux-musl/release/entrusted-webclient /src/entrusted_webserver/target/x86_64-unknown-linux-musl/release/entrusted-webserver /src/entrusted_client/target/x86_64-unknown-linux-musl/release/entrusted-cli /artifacts/entrusted-0.3.1-linux-amd64/ && mkdir -p /tmp/appdir/usr/bin /tmp/appdir/usr/share/icons && cp /ci_cd/linux/xdg/* /tmp/appdir/ && cd /src && cp /src/entrusted_client/target/x86_64-unknown-linux-gnu/release/entrusted-gui /tmp/appdir/ && cp /src/images/Entrusted_icon.png /tmp/appdir/usr/share/icons/entrusted-gui.png && ARCH=x86_64 linuxdeploy --appdir /tmp/appdir --desktop-file /tmp/appdir/entrusted-gui.desktop --icon-file /tmp/appdir/usr/share/icons/entrusted-gui.png --output appimage && mv *.AppImage /artifacts/entrusted-0.3.1-linux-amd64/entrusted-gui'
Trying to pull docker.io/uycyjnzgntrn/rust-linux:1.67.0...
Getting image source signatures
Copying blob 5f17f153f19d done  
Copying config 4a1dc90b2c done  
Writing manifest to image destination
Storing signatures
Error: crun: error stat'ing file `/dev/vboxusb/001/004`: Permission denied: OCI permission denied
WARN[0241] Failed to add pause process to systemd sandbox cgroup: Process org.freedesktop.systemd1 exited with status 1 
+ retVal=126
+ '[' 126 -ne 0 ']'
+ echo 'Failure to create Linux amd64 Linux binaries'
Failure to create Linux amd64 Linux binaries
+ exit 1

And I see:

ls -lah  /dev/vboxusb/001/004
crw-rw---- 1 root vboxusers 189, 3 May 30 14:39 /dev/vboxusb/001/004

Is this related to the podman run as root versus rootless issue ?

[yveszoundi]

This is likely due to the "--privileged" flag and implications for VirtualBox.

It's been a very long time since I used VirtualBox, but I recall groups such vboxusers or something among those lines. Is your user part of the vboxusers group?

If the above is not relevant or does not work, there's a suggestion in podman Github issues to append "---group-add keep-groups" to the problematic run command. The changes would be in the ci_cd/linux/build.sh file for the podman run command

[stef204]

btw, I no longer have a vboxusers group on my box; so is the script creating this /dev/vboxusb/001/004 ?

[yveszoundi]

The script doesn't create devices in "/dev" or whatever

[stef204]

+ podman run --platform linux/amd64 --rm --privileged --group-add keep-groups -v ......
Error: mkdir /run/containers: permission denied

And:

% sudo ls -lah /run/containers    
total 0
drwx------  4 root root   80 May 30 16:22 .
drwxr-xr-x 43 root root 1000 May 30 16:22 ..
drwxr-xr-x  2 root root   40 May 30 16:22 networks
drwx------  6 root root  120 May 30 16:22 storage

I don't think it should be creating this directory owned by root, especially inside /run.

NOTE: I tried to remove the - --privileged --group-add keep-groups and that still gives me the exact same error message Error: mkdir /run/containers: permission denied.

I did delete /run/containers before trying again.

This seems to be related to the vagaries of podman configuration, etc.

This is how it is on Arch; not sure if you can see anything there different which might create these issues.

[yveszoundi]

Few questions

• What is the value of the XDG_RUNTIME_DIR and XDG_DATA_DIR environment variables on your system?
• Are you running the build in a custom chroot like environment?
• Do you have several global configurations in /etc/containers/containers.conf?
  • I never needed more than few environment variables and maybe "podman system migrate" on Arch Linux in the past.
  • podman also looks at any user defined config files in ($XDG_CONFIG_HOME/.config/containers (if I recall correctly)

Not sure why the scripts are looking in /run/containers either...

• Typically they would look into the home folder of the current user something such as "~/.local/share/containers" unless your environment is special or you're running maybe processes as root?
• The only thing that is important in the run directory would be the default value of XDG_RUNTIME_DIR which is often "/run/$(id -u)" on systemd based Linux distributions such as arch and several others.

[stef204]

• What is the value of the XDG_RUNTIME_DIR and XDG_DATA_DIR environment variables on your system?

% echo $XDG_RUNTIME_DIR 
/run/user/1000

• Are you running the build in a custom chroot like environment?

No

• Do you have several global configurations in /etc/containers/containers.conf?

See:

% diff /usr/share/containers/containers.conf /etc/containers/containers.conf  
Files /usr/share/containers/containers.conf and /etc/containers/containers.conf are identical

• podman also looks at any user defined config files in ($XDG_CONFIG_HOME/.config/containers (if I recall correctly)
  Not sure why the scripts are looking in /run/containers either...

See here.
Because of line 20 ?
(I have modified line 34 but I don't see a problem with it.

• Typically they would look into the home folder of the current user something such as "~/.local/share/containers" unless your environment is special or you're running maybe processes as root?

Not running as root (that I know of, i.e. I am not using sudo, etc.)

[yveszoundi]

It seems that your podman instance is not configured properly for rootless mode per reference documentation. In rootless Podman both graphroot and runroot should be ignored.

The runroot setting should default to $XDG_RUNTIME_DIR/containers in rootless mode, if I'm not mistaken.

This is a subset of my /etc/containers/storage.conf file on my local build machine (Ubuntu virtual machine). Folders are not the same as yours, but my non-root user definitely doesn't have permissions to write data in the "/var" directory manually.

# Temporary storage location
runroot = "/var/run/containers/storage"

# Primary Read/Write location of container storage
graphroot = "/var/lib/containers/storage"

If you still encounter issues, I suggest checking this Arch Linux specific guide to ensure that it can run successfully in rootless mode. Sadly, I do not have an Arch Linux machine around anymore.

[stef204]

I would agree on the fact that podman is not configured properly for rootless mode.
As a quick test, I removed the --privileged from the command in build.sh and I get much further in the build process up until this new error:

fuse: device not found, try 'modprobe fuse' first
open dir error: No such file or directory
+ retVal=127
+ '[' 127 -ne 0 ']'
+ echo 'Failure to create Linux amd64 Linux binaries'
Failure to create Linux amd64 Linux binaries
+ exit 1

the fuse is loaded so not sure why it complains.

But overlay' is the driver, I have verified that using podman info`.

But I have to say that that was just a quick and dirty test and I should probably remove everything and start from scratch, following Arch's wiki and the guide you linked to.

It's going to take me some time -- and the iso you provide works inside a VM (provided you forward port 13000) so I am not extremely motivated to debug this.

NOTE: it might be a good idea to mention in the documentation that one has to forward port 13000 (and perhaps 22 if one wants to use ssh) for the live-cd iso to work properly and be reachable by the host.

(It's easy to do in Virtualbox [you check a box during VM creation and input the port number]; but I have no idea of how to do it for libvirt/virt-manager using regular user session.)

NOTE: Tried again and still same error:

   Compiling rpassword v7.2.0
   Compiling entrusted_l10n v0.3.0 (/src/entrusted_l10n)
   Compiling clap v4.3.0
   Compiling toml v0.7.4
    Finished release [optimized] target(s) in 1m 12s
fuse: device not found, try 'modprobe fuse' first
open dir error: No such file or directory
+ retVal=127
+ '[' 127 -ne 0 ']'
+ echo 'Failure to create Linux amd64 Linux binaries'
Failure to create Linux amd64 Linux binaries
+ exit 1

[stef204]

Good news. I've finally resolved the issue by adding the following option to podman run in build.sh : --group-add keep-groups.
See this.
It builds successfully now.

Onto the next error.
After building linux binaries successfully, when I cd into entrusted/ci_cd/live_cd/ to run build.sh to build the iso, it fails with:

+ RUST_CI_VERSION=1.67.0
+ echo '>>> Creating LIVE_BOOT folder'
>>> Creating LIVE_BOOT folder
+ test -d /tmp/entrusted-tmpbuild/entrusted-livecd/live_boot_dir-0.3.1-amd64
+ sudo rm -rf /tmp/entrusted-tmpbuild/entrusted-livecd/live_boot_dir-0.3.1-amd64
+ mkdir -p /tmp/entrusted-tmpbuild/entrusted-livecd/live_boot_dir-0.3.1-amd64
+ sudo chmod -R a+rw /tmp/entrusted-tmpbuild/entrusted-livecd/live_boot_dir-0.3.1-amd64
+ echo '>>> Boostraping Debian installation'
>>> Boostraping Debian installation
+ sudo debootstrap --arch=amd64 --variant=minbase bullseye /tmp/entrusted-tmpbuild/entrusted-livecd/live_boot_dir-0.3.1-amd64/chroot https://mirror.csclub.uwaterloo.ca/debian/
/usr/share/debootstrap/functions: line 1665: /tmp/entrusted-tmpbuild/entrusted-livecd/live_boot_dir-0.3.1-amd64/chroot/test-dev-null: Permission denied
E: Cannot install into target '/tmp/entrusted-tmpbuild/entrusted-livecd/live_boot_dir-0.3.1-amd64/chroot' mounted with noexec or nodev
+ sleep 10
+ sudo debootstrap --arch=amd64 --variant=minbase bullseye /tmp/entrusted-tmpbuild/entrusted-livecd/live_boot_dir-0.3.1-amd64/chroot https://mirror.csclub.uwaterloo.ca/debian/
/usr/share/debootstrap/functions: line 1665: /tmp/entrusted-tmpbuild/entrusted-livecd/live_boot_dir-0.3.1-amd64/chroot/test-dev-null: Permission denied
E: Cannot install into target '/tmp/entrusted-tmpbuild/entrusted-livecd/live_boot_dir-0.3.1-amd64/chroot' mounted with noexec or nodev
+ retVal=1
+ '[' 1 '!=' 0 ']'
+ echo 'Could not bootstrap Debian installation!'
Could not bootstrap Debian installation!
+ exit 1
+ retVal=1
+ '[' 1 '!=' 0 ']'
+ echo 'Failed to prepare build for amd64'
Failed to prepare build for amd64
+ exit 1

This looks like a permission error or different mount options for /tmp ?

Edit:
This error gets fixed by remounting /tmp without the nosuid,nodev options.
So, I used sudo mount -o remount,exec,dev /tmp

That worked.

[stef204]

Next error:

>>> Building container image
+ cd /mnt/ssdd/Downloads/src/entrusted/app
+ podman build --squash-all --jobs 2 --platform linux/amd64 -t docker.io/uycyjnzgntrn/entrusted_container:0.3.1 -f entrusted_container/Dockerfile .
[1/2] STEP 1/5: FROM docker.io/uycyjnzgntrn/debian:bullseye-rust-1.64.0-tesseract5 AS builder
[2/2] STEP 1/18: FROM docker.io/debian:bullseye-slim
[2/2] STEP 2/18: LABEL maintainer="Yves Zoundi, Rimero Solutions Inc."
[2/2] STEP 3/18: LABEL org.opencontainers.image.title="Entrusted document sanitizer container image"
[2/2] STEP 4/18: LABEL org.opencontainers.image.description="Conversion of potentially suspicious documents to trusted PDFs, inside a container"
[2/2] STEP 5/18: LABEL org.opencontainers.image.authors="Yves Zoundi"
[2/2] STEP 6/18: LABEL org.opencontainers.image.vendor="Rimero Solutions Inc."
[2/2] STEP 7/18: LABEL org.opencontainers.image.documentation="https://github.com/rimerosolutions/entrusted/tree/main/app/entrusted_container"
[2/2] STEP 8/18: LABEL org.opencontainers.image.licenses="GPLv3"
[2/2] STEP 9/18: LABEL org.opencontainers.image.version="0.3.1"
[2/2] STEP 10/18: LABEL org.opencontainers.image.url="https://github.com/rimerosolutions/entrusted/tree/main/app/entrusted_container"
[2/2] STEP 11/18: LABEL org.opencontainers.image.source="https://github.com/rimerosolutions/entrusted.git"
[2/2] STEP 12/18: RUN echo "deb [trusted=yes] https://notesalexp.org/tesseract-ocr5/bullseye/ bullseye main" >> /etc/apt/sources.list     && DEBIAN_FRONTEND=noninteractive apt-get -o "Acquire::Retries=10" -o "Acquire::https::Verify-Peer=false" update     && DEBIAN_FRONTEND=noninteractive apt-get -o "Acquire::https::Verify-Peer=false" install --no-install-recommends -y     libleptonica-dev     libtesseract-dev     libreoffice-writer     libreoffice-draw     libreoffice-impress     libreoffice-calc     libpoppler-dev     libcairo2-dev     tesseract-ocr-all     fonts-dejavu     libpoppler-glib-dev     && apt-get clean
[1/2] STEP 2/5: LABEL stage=entrusted_container_builder
[1/2] STEP 3/5: WORKDIR /home/rust/entrusted-container
[1/2] STEP 4/5: COPY . .
error running container: from /usr/bin/crun creating container for [/bin/sh -c echo "deb [trusted=yes] https://notesalexp.org/tesseract-ocr5/bullseye/ bullseye main" >> /etc/apt/sources.list     && DEBIAN_FRONTEND=noninteractive apt-get -o "Acquire::Retries=10" -o "Acquire::https::Verify-Peer=false" update     && DEBIAN_FRONTEND=noninteractive apt-get -o "Acquire::https::Verify-Peer=false" install --no-install-recommends -y     libleptonica-dev     libtesseract-dev     libreoffice-writer     libreoffice-draw     libreoffice-impress     libreoffice-calc     libpoppler-dev     libcairo2-dev     tesseract-ocr-all     fonts-dejavu     libpoppler-glib-dev     && apt-get clean]: sd-bus call: Process org.freedesktop.systemd1 exited with status 1: Input/output error
: exit status 1
ERRO[0001] did not get container create message from subprocess: EOF 
[1/2] STEP 5/5: RUN CARGO_NET_RETRY=10 CARGO_NET_GIT_FETCH_WITH_CLI=true PKG_CONFIG_ALLOW_CROSS=1 LO_INCLUDE_PATH=/usr/include/LibreOfficeKit     cargo install --path entrusted_container --root /usr/local     && cargo clean --manifest-path entrusted_container/Cargo.toml     && strip --strip-unneeded /usr/local/bin/entrusted-container
Error: building at STEP "RUN echo "deb [trusted=yes] https://notesalexp.org/tesseract-ocr5/bullseye/ bullseye main" >> /etc/apt/sources.list     && DEBIAN_FRONTEND=noninteractive apt-get -o "Acquire::Retries=10" -o "Acquire::https::Verify-Peer=false" update     && DEBIAN_FRONTEND=noninteractive apt-get -o "Acquire::https::Verify-Peer=false" install --no-install-recommends -y     libleptonica-dev     libtesseract-dev     libreoffice-writer     libreoffice-draw     libreoffice-impress     libreoffice-calc     libpoppler-dev     libcairo2-dev     tesseract-ocr-all     fonts-dejavu     libpoppler-glib-dev     && apt-get clean": while running runtime: exit status 1
+ sleep 10
+ podman build --squash-all --platform linux/amd64 -t docker.io/uycyjnzgntrn/entrusted_container:0.3.1 -f entrusted_container/Dockerfile .
[1/2] STEP 1/5: FROM docker.io/uycyjnzgntrn/debian:bullseye-rust-1.64.0-tesseract5 AS builder
[1/2] STEP 2/5: LABEL stage=entrusted_container_builder
[1/2] STEP 3/5: WORKDIR /home/rust/entrusted-container
[1/2] STEP 4/5: COPY . .
[1/2] STEP 5/5: RUN CARGO_NET_RETRY=10 CARGO_NET_GIT_FETCH_WITH_CLI=true PKG_CONFIG_ALLOW_CROSS=1 LO_INCLUDE_PATH=/usr/include/LibreOfficeKit     cargo install --path entrusted_container --root /usr/local     && cargo clean --manifest-path entrusted_container/Cargo.toml     && strip --strip-unneeded /usr/local/bin/entrusted-container
error running container: from /usr/bin/crun creating container for [/bin/sh -c CARGO_NET_RETRY=10 CARGO_NET_GIT_FETCH_WITH_CLI=true PKG_CONFIG_ALLOW_CROSS=1 LO_INCLUDE_PATH=/usr/include/LibreOfficeKit     cargo install --path entrusted_container --root /usr/local     && cargo clean --manifest-path entrusted_container/Cargo.toml     && strip --strip-unneeded /usr/local/bin/entrusted-container]: sd-bus call: Process org.freedesktop.systemd1 exited with status 1: Input/output error
: exit status 1
ERRO[0002] did not get container create message from subprocess: EOF 
Error: building at STEP "RUN CARGO_NET_RETRY=10 CARGO_NET_GIT_FETCH_WITH_CLI=true PKG_CONFIG_ALLOW_CROSS=1 LO_INCLUDE_PATH=/usr/include/LibreOfficeKit     cargo install --path entrusted_container --root /usr/local     && cargo clean --manifest-path entrusted_container/Cargo.toml     && strip --strip-unneeded /usr/local/bin/entrusted-container": while running runtime: exit status 1
+ podman image prune -f --filter label=stage=entrusted_container_builder
+ podman save -m -o /tmp/entrusted-tmpbuild/entrusted-livecd/live_boot_tmpdir-0.3.1-amd64/image.tar docker.io/uycyjnzgntrn/entrusted_container:0.3.1
Error: docker.io/uycyjnzgntrn/entrusted_container:0.3.1: image not known
+ retVal=125
+ '[' 125 '!=' 0 ']'
+ echo 'Unable to export container image to tar archive!'
Unable to export container image to tar archive!
+ exit 1
+ retVal=1
+ '[' 1 '!=' 0 ']'
+ echo 'Failed to prepare build for amd64'
Failed to prepare build for amd64
+ exit 1

Here I am stumped.
I think this entire building of the ISO should occur in a VM anyway, like Ubuntu.

I especially dislike the fact that the build script for the iso actually creates a brand new user AND group (non-system, with an actual home /home/entrusted, etc.) I do not want this on my system honestly.

[yveszoundi]

I especially dislike the fact that the build script for the iso actually creates a brand new user AND group (non-system, with an actual home /home/entrusted, etc.) I do not want this on my system honestly.

• The entrusted user is forcefully deleted at the end and beginning of builds. That seemed to be the only working option for multi-arch builds (amd64 and arm64): several podman/docker errors that I also found in open podman issues.
• I tried several times to fully bootstrap the entire process inside the temporary chroot environment or podman/docker
• What worked was building locally and then copying local "podman container folders" owned by the entrusted users to the ISO. I'm open to pull requests though, if there are better working options.

Next error:

From the error messages log, it seems that the container image is not built properly: The ISO scripts copy the container files to the "ISO image layer" within a chroot environment.

• error running container: from /usr/bin/crun creating container for.....
• sd-bus call: Process org.freedesktop.systemd1 exited with status 1: Input/output error

The error itself appears to be related to dbus.

• is dbus installed and running on your system? dbus-run-session or similar executable (ps auxww|grep -i dbus)
• if using a custom window manager or using startx or similar, are you bootstrapping it with dbus launch scripts?

I think this entire building of the ISO should occur in a VM anyway, like Ubuntu.

On my system, I always build locally from an Ubuntu virtual machine (QEMU). "Recently", I ported the local build process to GitHub actions and that follows closely what's happening on a local Ubuntu VM.

• Linux distribution packaging differences and possibly configuration differences (systemd services, config files different or user-customized for various reasons such as disk storage preferences, etc.)
• User technical knowledge, specific Linux knowledge and resourcefulness challenges
• I'm not a docker/podman expert either and I run a pretty vanilla podman configuration except when experimenting with runtime engines.

[stef204]

The user entrusted did not get deleted because the build script failed prior to completion.
I do have DBUS running on my system.
This should be built in a VM -- let me set one up with Ubuntu and give it a shot there.
Will post back.
Thanks.