WIP: btrfs-subvol support impermanence on provision #20
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The 6.11 release of btrfs-progs will be able to make subvols during mkfs. Take advantage of that so as to not need to do a bunch of filesystem manipulation during first-boot, and remove the need to mount the filesystem on boot to inspect it on every boot. The inline btrfs-progs override is horrible, but it will probably live less than a month, and I do not want to override the btrfs-progs in nixpkgs. For now the subvols are hard-coded, as they previously were with RAMIFY. The hard-coding will be removed in a follow up.
To allow directly provisioning to impermanence, / cannot be used for files that are needed on-boot. Move nix-path-registration into /nix when creating a FS with btrfs-subvol.
nakato
force-pushed
the
DirectImpermanence2
branch
from
d513ba4 to
2d86b25
Compare
|
Amazing. Can't wait to test that stuff. I'm still waiting for hardware. |
By creating subvolumes based on config.fileSystem, a root subvolume is not created if it is not defined, allowing impermanence. Extra subvolumes that need to exist, such as for persistence can be created, allowing impermanence.
Fakeroot was not taking effect, resulting in all files being owned by UID 1000 and GID 100. Use unshare to enter a user namespace with the user mapped as root such that the files loaded into the btrfs filesystem are owned by the root user and thus fixing multiple boot issues related to permissions and security. Namely, systemd-tmpfiles refuses to operate on a number of paths and sshd refuses to accept pubkeys due to insecure permissions in path. The nix-store also would be editable by UID 1000, and maybe members of GID 100. The nix-daemon eventually cleans this up, but should never be this way.
nakato
force-pushed
the
DirectImpermanence2
branch
from
2d86b25 to
30f9529
Compare
|
This has been merged in merge-commit 987002c |
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently you have to boot the system once before you can convert it to impermanence, manually creating your persistent data subvol and then after switching to it, deleting the
@subvol.This is suboptimal, and it should be possible to directly provision with impermanence.
First up, the RAMIFY tooling is annoying, and now that btrfs-progs will (soon) support subvol creation during mkfs, throw that all away and use it instead, removing the risk of the metadata allocation filling up during the shuffle and throwing ENOSPC. As the in-line override is only used during the build of the sdImage, it's contained away from everything else enough I'm happy to do this for now, I expect it'll be able to be removed in a month.
Second with impermanence, / won't be around, so we need to pick someplace that will be persistent to hold nix-path-registration. All of /nix is probably persistent, and I don't really want to stuff it into /nix/store, so into /nix it goes. It's a bootstrap detail, so it's created during sdImage creation and used on first-boot, we can change its location however we want without breaking live systems.
Finally, we work out the subvolMap passed to the forked make-btrfs-fs from config.system.build.fileSystems(?) so
/can be a tmpfs. It could use some more asserts for sanity checking, but is otherwise ready to go.Last patch is not boot tested yet, as the first pass missed re-sizing, so I need to re-run that.
Ref: #18