podman-rootless

Example of using nix + flakes to have podman rootless working

nix flake clone github:ES-Nix/podman-rootless --dest podman-rootless

nix develop github:ES-Nix/podman-rootless/324855d116d15a0b54f33c9489cf7c5e6d9cd714 --command ./install-podman.sh && ./test_podman-rootless.sh

nix develop github:ES-Nix/podman-rootless/bffe8ae0d5b933b321e9fc0de25d992f5f5540d0

Install via git

git clone https://github.com/ES-Nix/podman-rootless.git
cd podman-rootless
git checkout bffe8ae0d5b933b321e9fc0de25d992f5f5540d0
nix develop

IMAGE_VERSION='localhost/nix-oci-dockertools:0.0.1'

podman run
--interactive
--rm=true
--tty=true
--workdir /code
--volume "$(pwd)":/code
"$IMAGE_VERSION" bash -c "sudo ls -al && id"

Other somehow hard tests:

podman \
run \
--interactive \
--rm=true \
--tty=true \
python:3.8 \
bash

podman \
run \
--interactive \
--rm=true \
--tty=true \
blang/latex \
bash

podman \
run \
--interactive \
--rm=true \
--tty=true \
wernight/funbox \
nyancat

Notes

The behavior "the nix develop command which uses the devShell.${system} flake output if it exists or defaultPackage.${system} otherwise.", source, is really important to understand the current working state.

Additional groups in buildFHSUserEnv TL;DR it looks like (i am not sure) it is not possible.

nix build \
&& result/fsh-podman-rootless-env podman --help

Troubleshooting

stat $(which newuidmap)
stat $(which newgidmap)

cat /proc/self/uid_map podman unshare cat /proc/self/uid_map

whereis newuidmap whereis newgidmap

ls "$HOME"/.config/containers ls "$HOME"/.local/share/containers

ls ~/.config/containers ls ~/.local/share/containers

rm -rf ~/.config/containers ~/.local/share/containers

Use the --log-level=debug, really usefull!

podman unshare cat /proc/self/uid_map from Meaning of this in containers/podman#3890 (comment)

filecap /usr/bin/newuidmap filecap $(which newuidmap) TODO: not tested

ls -l /usr/bin/new{uid,gid}*

TODO: reproduce it using QEMU? containers/podman#3890 (comment) containers/podman#3890 (comment)

UID_INSIDE=$(podman run --name UID_probe --rm foo-image /usr/bin/id -u)
podman unshare chown -R $UID_INSIDE volumes

podman run --pod foo-pod --name foo\
 --rm\
 -v $VOLUMES/data:$CONTAINER/data\
 foo-image

containers/podman#7778 (comment)

TODO: important! NixOS/nixpkgs#112902

About the profile in the buildFHSUserEnv, gsc.io sec-fhs-environments https://github.com/NixOS/nixpkgs/pull/80457/files#diff-aff959a600d3441934b3b905339c0f90dcd8122e8774ee2dbcae35d72f349991R152

IHaskell + jupyter + notebook + buildFHSUserEnv https://vaibhavsagar.com/blog/2018/03/17/faking-non-nixos-stack/

I've downloaded a binary, but I can't run it, what can I do? buildFHSUserEnv https://nixos.wiki/wiki/FAQ#How_can_I_manage_software_with_nix-env_like_with_configuration.nix.3F

TODO: what is this? https://discourse.nixos.org/t/setting-run-user-with-oci-containers-and-systemd/9900/8

Faced a annoying behavior:

The podman command was in path, and it must not be, so a did:

which podman

nix-env --query | cat

nix-env --uninstall podman-wrapper-2.1.1

A improved version: nix-env --query --installed --out-path | cat

podman unshare cat /proc/self/uid_map If this only shows 1 line, then you have not setup /etc/subuid and /etc/subgid properly or your newuidmap and newgidmap tools are not install properly. from

WIP with

git clone https://github.com/ES-Nix/podman-rootless.git
cd podman-rootless
git checkout X

nix develop

Why sudo --preserve-env su -c 'nix develop' prints:

bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell
Entering the nix devShell
bash: cannot set terminal process group (14581): Inappropriate ioctl for device
bash: no job control in this shell

Why even using nix develop --ignore-environment the docker binary still in path? See readlink $(which docker).

TODO: maybe it is the problem?

ls /etc/cni/net.d/ ls /opt/cni/bin

containers/podman#3679 (comment)

ls /nix/store/* | grep cni-

sudo podman --log-level=debug images

dpkg-query -L podman Incompatibilities of podman from docker on Travis CI

Probably the one of the problems, missing this file: https://github.com/containers/podman/tree/master/cni

Use something like this to test the CNI: podman run --network foo --rm -it alpine ls containers/podman#2909 (comment) containernetworking/cni#770 (comment)

TODO: how to check it? ip link add cni-podman0 type bridge containers/podman#4114 (comment)

Credits and history

TODO: improve it, i am busy trying to make it work first.

• While searching for some problem that i was facing i have found this issue comment from adisbladis, it was pointing to a gist that he have done. I didn't test it in NixOS, but was able to use the nix-shell (it was intended to be used as nix-shell) and tranform it in a flake and do some crazy stuff to combine it in other flakes take a look at the example of nix, flakes, shellHook, writeShellScriptBin, defaultPackage, all together that uses what i did in this rev 170f002d76070b1d281cf7e6868076bcfb1fea07. But a faced a problem, the file system, yes, even this kind of stuff to make things break. Podman was working really ok, but when i tried to load a "big" OCI image with size > 0.5Gbyte it broke. The podman mantainers say "We recommend using fuse-overlayfs instead, as it is capable of deduplicating storage." So now i am trying to use buildFHSUserEnv, definition in nixpokgs buildFHSUserEnv to solve it adapting the danieldk commented.
• https://www.youtube.com/watch?v=RDzsrmMl48I

About Filesystem Hierarchy Standard (FHS)

• Excelent: On Nix, NixOS and the Filesystem Hierarchy Standard (FHS)

• Sander van der Burg is the creator of buildFHSUserEnv, must read: Composing FHS-compatible chroot environments with Nix (or deploying Steam in NixOS)

• Podman official documentation: Unsupported file systems in rootless mode

• Maintainers in the podman repository "we recommend using fuse-overlayfs"

• Rootless Containers

• YouTube ExplainingComputers: Explaining File Systems: NTFS, exFAT, FAT32, ext4 & More

• YouTube Joe Collins: Learning the Linux File System

• YouTube EuroBSDCon2014: FUSE and beyond: bridging filesystems by Emannuel Dreyfus

• YouTube The Linux Man: Linux File System Types

• developer.ibm Anatomy of ext4

• Linux Filesystems: Where did they come from? [linux.conf.au 2014]

• TODO watch it A Study of Linux File System Evolution

• TODO find scientific papers that go even more deeper in all this Understanding Linux filesystems: ext4 and beyond

• TODO replicate it using flakes Making a Simple Deb Package NixOS Compatible (Mathematica's wolframscript)

RAID 0, RAID 1, RAID 10 - All You Need to Know as Fast As Possible, it looks like it is really old and SSDs have changed it all RAID 5 & RAID 6 - All You Need to Know as Fast As Possible.

Explains about history in the beginning: btrfs: The Best Filesystem You've Never Heard Of Deploying Btrfs at Facebook Scale - Josef Bacik, Facebook

File Systems | Which One is the Best? ZFS, BTRFS, or EXT4

All File Systems Are Not Created Equal: On the Complexity of Crafting Crash-Consistent Applications

TODO: Try to make it work:
https://discourse.nixos.org/t/build-a-yocto-rootfs-inside-nix/2643/22

TODO: Find the refs it cites In-depth: ELF - The Extensible & Linkable Format and find an example of hardcoded path in the ELF and make from zero one working example. 2013 Day2P18 LoB: ELF Intro 1 Handmade Linux x86 executables: ELF header and 2 Handmade Linux x86 executables: Hello, world.

Running as root

sudo \
--preserve-env \
su \
--preserve-env \
root \
-c 'nix develop --ignore-environment'

su \
--preserve-env \
pedro \
-c 'echo 123 | sudo --stdin podman images'

Why the --login gives problems? It somehow scruds with the terminal!

0b4d0714bfaab2d3fd45176699658c1ae5437742

git clone https://github.com/ES-Nix/podman-rootless.git
cd podman-rootless
git checkout 0b4d0714bfaab2d3fd45176699658c1ae5437742
nix develop

sudo
--preserve-env
su
--preserve-env
root
-c 'nix develop --ignore-environment github:ES-Nix/podman-rootless'