Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

grub-btrfs.path for timeshift is not working anymore #211

Closed
pedrohqb opened this issue Jun 27, 2022 · 26 comments
Closed

grub-btrfs.path for timeshift is not working anymore #211

pedrohqb opened this issue Jun 27, 2022 · 26 comments

Comments

@pedrohqb
Copy link

grub-btrfs.path is not working anymore as /run/timeshift/backup is not used anymore to mount the snapshots directory, but rather a random numeric folder inside /run/timeshift, such as "/run/timeshift/35373/backup/"
It seems to be a change made by the Linux Mint team.

@Schievel1
Copy link
Collaborator

Hm since there are two timeshifts now (https://github.com/teejee2008/timeshift, https://github.com/linuxmint/timeshift) no one seemed to notice. But this breaking change is going to be introduced into the original version (the first link) sooner or later.

@pedrohqb
Copy link
Author

pedrohqb commented Jun 28, 2022

There are some people complaining about the same problem: https://forum.manjaro.org/t/grub-doesnt-list-my-manual-btrfs-timeshift-snapshots-anymore/113618/5

As a temporary workaround, I am using this: #138 (comment)

@JimPix1
Copy link

JimPix1 commented Jul 15, 2022

Yeah, I'm having this trouble too

@LorenzoBettini
Copy link

I confirm that the workaround proposed above works for me (in case, I've also blogged about that https://www.lorenzobettini.it/2022/07/timeshift-and-grub-btrfs-in-linux-arch/)

@JimPix1
Copy link

JimPix1 commented Jul 16, 2022

Very nice guide! Although it doesn't work properly for me. In order for grub-btrfs to refresh the snapshots I must update grub. grub-btrfs doesn't automatically monitor the snapshots and update accordingly

@JimPix1
Copy link

JimPix1 commented Jul 16, 2022

Fixed the above with this comment: #138 (comment)

@LorenzoBettini
Copy link

LorenzoBettini commented Jul 16, 2022

Very nice guide! Although it doesn't work properly for me. In order for grub-btrfs to refresh the snapshots I must update grub. grub-btrfs doesn't automatically monitor the snapshots and update accordingly

@JimPix1
Are you sure you followed the steps and enabled grub-btrfs.path

@JimPix1
Copy link

JimPix1 commented Jul 16, 2022

I did. Obviously timeshift-autosnap will regenerate grub for me, but outside of autosnap, any other snapshots made don't appear unless you update grub

@LorenzoBettini
Copy link

Then grub-btrfs is not configured correctly..

@JimPix1
Copy link

JimPix1 commented Jul 16, 2022

It definitely is, You need a oneshot service to regenerate grub on shutdown or reboot in order for it to always update. Either that or do it yourself or just let timeshift-autosnap do it for you. Grub-btrfs does not regenerate grub for you.

@LorenzoBettini
Copy link

Ok, you have to update grub once and for all (so that the submenu for snapshots it's added) and then grub-btrfs simply automatically regenerates its submenu from then on, as soon as a snapshot appears in the configured directory. If that doesn't happen you have a problem in it's configuration. At least that's how it's supposed to work and how it works for me.

@JimPix1
Copy link

JimPix1 commented Jul 16, 2022

Odd, it doesn't automatically regenerate it's submenu for me, and I didn't change the grub-btrfs configuration.
The service script does the job at least and the submenu is always up-to-date when I power my laptop back on.

@LorenzoBettini
Copy link

That's the point: you have to change its configuration since, by default it looks for snapshots in /.snapshots, while timeshift generates in another one.

@JimPix1
Copy link

JimPix1 commented Jul 16, 2022

Yes, I changed that already

@LorenzoBettini
Copy link

Then I wouldn't know.. as I said, it works for me

@Schievel1
Copy link
Collaborator

Schievel1 commented Aug 23, 2022

I think the important information here is that the name of this directory in /run/timeshift is not just a random number but the PID of the currrently running timeshift. See https://github.com/linuxmint/timeshift/blob/829c78aa072ac06bf04f1f0ae865b4fcf500bc7d/src/Core/Main.vala#L176
(Watching this line now it is strange that Lorenzos workaround works at all and timeshift uses /run/timeshift/backup/..., because that line 176 from the link isn't conditional. So there must be some mechanism in timeshift I don't know about yet)

For OpenRC I simply watch /run/timeshift/ for a folder to be created via inotify. If its created I know timeshift is started and I look with ps and awk (there is also pgrep, but I did not want to introduce another dependency) for the PID of timeshift to create a snapshot path to watch for this session.
I wonder if that is also possible with systemd, sadly my knowledge about systemd is rather small. I don't know how to dynamically change the path in grub-btrfs.path and if thats even possible.

@Schievel1
Copy link
Collaborator

Schievel1 commented Aug 29, 2022

Guys, I wanted to discuss a thing with you. Since we wrote that openrc daemon that is using inotify to fix this issue, I thought maybe we could use this daemon also for the systemd service, instead of depending on systemd mount units.
I tested a quick draft on a old laptop I set up with systemd.

So, basically have a look at that: https://github.com/Schievel1/grub-btrfs/tree/systemd-inotify
Important changes are in the grub-btrfs.service:

[Unit]
Description=Regenerate grub-btrfs.cfg

[Service]
Type=simple
# Set the possible paths for `grub-mkconfig`
Environment="PATH=/sbin:/bin:/usr/sbin:/usr/bin"
# Load environment variables from the configuration
EnvironmentFile=/etc/default/grub-btrfs/config
# Regenerate just '/boot/grub/grub-btrfs.cfg' if it exists and is not empty, else regenerate the whole grub menu
ExecStart=/usr/bin/grub-btrfs-daemon

This service is now just starting a script as a daemon.

And the daemon itself, which goes into /usr/bin:

#!/bin/sh
# Copyright 2022 Pascal Jaeger
# Distributed under the terms of the GNU General Public License v3
# Update GRUB when new BTRFS snapshots are created.

timeshift_pid
watchtime=0

echo "im running" >> /home/pascal/daemontest
snapshots="/.snapshots"
timeshift_auto="true"

if  ! [ "${#snapshots}" -gt 0 ] && ! [ ${timeshift_auto} ]; then
    echo "Please specify the snapshots directory" >&2
    exit 1
fi
if [ ${timeshift_auto} ]; then
    watchtime=15
else
    watchtime=0
fi

echo "entering while" >> /home/pascal/daemontest
while true; do
    if [ ${timeshift_auto} == "true" ] && ! [ "${#timeshift_pid}" -gt 0 ] ; then
        inotifywait -e create -e delete /run/timeshift && {
            echo "pid: ${timeshift_pid}" >> /home/pascal/daemontest
            echo "snapshots: ${snapshots}" >> /home/pascal/daemontest
            sleep 1
            timeshift_pid=$(ps ax | awk '{sub(/.*\//, "", $5)} $5 ~ /timeshift/ {print $1}')
            snapshots="/run/timeshift/${timeshift_pid}/backup/timeshift-btrfs/snapshots"
        }
    else
        while [ -d "$snapshots" ]; do
            sleep 1
            inotifywait -e create -e delete -e unmount -t "$watchtime" "$snapshots" && {
                echo "got shot" >> /home/pascal/daemontest
                sleep 5
                if [ -s "${GRUB_BTRFS_GRUB_DIRNAME:-/boot/grub}/grub-btrfs.cfg" ]; then
                    /etc/grub.d/41_snapshots-btrfs
                else
                    ${GRUB_BTRFS_MKCONFIG:-grub-mkconfig} -o ${GRUB_BTRFS_GRUB_DIRNAME:-/boot/grub}/grub.cfg
                fi
            }
        done
        timeshift_pid=""
    fi
done

Keep in mind, this I just a quick draft to see if this works at all. The variables should later not be hardcoded of course. I also left the echos in there for debugging. If you want to test this out, change /usr/lib/grub-btrfs.service accordingly and put this script into /usr/bin and make it executeable.
This works for me as the openrc daemon where this is from is working as well.

So if someone with more knowledge in systemd could help me out here (@Kr1ss-XD, @LorenzoBettini) :

  • What do you think about this? I have no experience at all in systemd, is this how such a thing should be solved, is this complete garbage or do you have a more suitable idea?
  • If this is somewhat reasonable:
    In openrc those variables like snapshots go in /etc/conf.d, a simple file where things are defined like this: snapshots="/.snapshots" . Then there would be a file in /etc/init.d that is basically like the .servic-file in systemd. It has a line like this command_args="${snapshots} ${timeshift_auto}", which gives the daemon the variables from the conf.d-File as arguments. Then in the script i would use snapshots_dir=${1} to use that variable. So how do I do that in systemd? It would be great to use the same daemon and same file in conf.d in the systemd version of this and only have the .service file different. I noticed that there are similar files in /etc/conf.d on my new systemd laptop.
  • is the .service-File doing the right thing? I tried Type=simple because, well, I thought its simple. I noticed there is also exec and idle.

@Schievel1
Copy link
Collaborator

To answer the second point:
https://wiki.gentoo.org/wiki/Project:Systemd/conf.d_files

Apparently using the same config file is a no no.
It also says

Referring to conf.d files in unit files is prohibited (see: conf.d files). Using other files for configurable variables is strongly discouraged. Instead, the unit file should use defaults, and a boilerplate /etc/systemd/system/foo.service.d/gentoo.conf with commented out additional options may be installed. It is encourage to just override ExecStart= there rather than invent a special environment variable for command-line options.

So I have to make a file in /etc/systemd/system/grub-btrfs.service.d/grub-btrfd.conf where I override the ExecStart somehow.

Well alright, I think I could at least reuse the daemon itself then

@danboid
Copy link
Contributor

danboid commented Aug 30, 2022

I have got it working now - systemd now auto updates GRUB when I create or delete timeshift snapshots so I have created a guide to getting this configured correctly.

Configuring systemd to update grub after modifying timeshift snapshots

Neither timeshift nor grub-btrfs will update your grub menu after you create or delete any snapshots so we have to manually configure systemd to do this for us, if we want the ability to boot directly into snapshots from GRUB.

First, we must check the base path used to store your timeshift snapshots. Create at least one snapshot with timeshift and then run timeshift --list

# timeshift --list
Mounted '/dev/sda5' at '/run/timeshift/6724/backup'
Device : /dev/sda5
UUID   : fac5f415-21f5-44e9-93a8-c4b6b5687c72
Path   : /run/timeshift/6724/backup
Mode   : BTRFS
Status : OK
1 snapshots, 387.5 GB free

Num     Name                 Tags  Description  
------------------------------------------------------------------------------
0    >  2022-08-30_21-56-43  O

We can see here that the path of the snapshot is /run/timeshift/6724/backup. If you create more snapshots you will notice a similar path used but with a different 4 digit number so from this we can see it is actually /run/timeshift that we want to monitor for new directories / snapshots.

systemd has a function called PathModified that we'll use to monitor /run/timeshift for changes to update grub but we can only use it with dynamically created dirs such as this by creating a new mount point for it. Edit /etc/fstab and add a line like:

UUID=<UUID> /run/timeshift/       btrfs   defaults,noatime  0 0

You must replace <UUID> with the UUID of your btrfs disk that you are snapshotting. After adding the new fstab entry, reboot and run:

# systemctl list-units -t mount
  UNIT                                              LOAD   ACTIVE SUB     DESCRIPTION                              
  ...
  run-timeshift.mount                               loaded active mounted /run/timeshift

To check systemd has registered the new mount point.

Create /lib/systemd/system/update-grub.service with the following contents, after checking grub-mkconfig -o /boot/grub/grub.cfg is the correct command to update GRUB on your OS. If not, adjust it as required:

[Unit]
Description=Regenerate grub menu

[Service]
Type=oneshot
# Regenerate the grub menu
ExecStart=bash -c 'grub-mkconfig -o /boot/grub/grub.cfg' 

Edit (or create) /lib/systemd/system/grub-btrfs.path with the following content. Replace every instance of run-timeshift.mount with the name of the systemd mount unit name used for the snapshots if the path / unit name differs on your system:

[Unit]
Description="Monitor /run/timeshift for new snapshots"
DefaultDependencies=no
Requires=run-timeshift.mount
After=run-timeshift.mount
BindsTo=run-timeshift.mount

[Path]
PathModified=/run/timeshift
Unit=update-grub.service

[Install]
WantedBy=run-timeshift.mount

Now by enabling / starting grub-btrfs.path, systemd will update GRUB when snapshots are created or deleted.

systemctl enable grub-btrfs.path
systemctl start grub-btrfs.path

@Schievel1
Copy link
Collaborator

Schievel1 commented Aug 31, 2022

Glad you got it working now.

it’s kind of a hack to make users fiddle around with their mount points in /etc/fstab.
and putting altering fstab in the Makefile with sed or so will certainly go against QA of most distros.

What do you think about my previous post, using the daemon we created for openrc with systemd?

@danboid
Copy link
Contributor

danboid commented Aug 31, 2022

@Schievel1

I'd not noticed your openrc solution before I had posted mine. A cross init solution would be good but my systemd solution is much simpler for systemd users to read and understand so I think I'll be sticking with my solution but I'd be tempted to switch to yours if it got more testing, dropped the debug statements and got included in grub-btrfs by default.

I couldn't see where in @LorenzoBettini 's wiki guide for setting this up with systemd it ever called grub-mkconfig, which was the main reason it was never going to work on my system, apart from the base snapshot path being wrong for my OS (Debian Testing). You'll notice in my guide Unit=update-grub.service has been added to the [PATH] section of grub-btrfs.path. There was no such trigger in the other examples and I don't understand any need for checking for pre-existing grub-btrfs config as grub-btrfs will regenerate this anyway when grub-update is run.

@Schievel1
Copy link
Collaborator

I am far from telling you what to do, but I think you can spare yourself the effort for that PR. I'm working on the daemon for both, systemd and openrc.

@danboid
Copy link
Contributor

danboid commented Aug 31, 2022

You seem confident this repo isn't dead then?

Did I miss anything from my revision of the systemd scripts?

@Schievel1
Copy link
Collaborator

Apparently it isn't #217
What do you mean you missed something? Sorry, I can not follow you.

@LorenzoBettini
Copy link

@JimPix1 @danboid I can't tell you why the steps in my tutorial do not work for you... I'm using that in 4 installations (Arch and EndeavourOS), and they work perfectly.

My suspect is that I'm using also timeshit-autosnap (as described in the tutorial)... maybe it works when timeshift-autosnap kicks in while it doesn't work if you create snapshots from the Timeshift GUI?

Here's some log during an update, and you can see that the grub menu is correctly generated with the snapshot just created:

...
(41/41) checking available disk space              [----------------------] 100%
:: Running pre-transaction hooks...
(1/2) Creating Timeshift snapshot before upgrade...
Using system disk as snapshot device for creating snapshots in BTRFS mode
Mounted '/dev/nvme0n1p13' at '/run/timeshift/3286/backup'
Creating new backup...(BTRFS)
Saving to device: /dev/nvme0n1p13, mounted at path: /run/timeshift/3286/backup
Created directory: /run/timeshift/3286/backup/timeshift-btrfs/snapshots/2022-09-01_09-55-22
Created subvolume snapshot: /run/timeshift/3286/backup/timeshift-btrfs/snapshots/2022-09-01_09-55-22/@
Created control file: /run/timeshift/3286/backup/timeshift-btrfs/snapshots/2022-09-01_09-55-22/info.json
BTRFS Snapshot saved successfully (0s)
Tagged snapshot '2022-09-01_09-55-22': ondemand
...
Found stale mount for device '/dev/nvme0n1p13' at path '/run/timeshift/3427/backup'
Unmounted successfully
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-linux-lts
Found initrd image: /boot/intel-ucode.img /boot/initramfs-linux-lts.img
Found fallback initrd image(s) in /boot:  intel-ucode.img initramfs-linux-lts-fallback.img
Warning: os-prober will not be executed to detect other bootable partitions.
Systems on them will not be added to the GRUB boot configuration.
Check GRUB_DISABLE_OS_PROBER documentation entry.
Adding boot menu entry for UEFI Firmware Settings ...
Detecting snapshots ...
Found snapshot: 2022-09-01 09:55:22 | timeshift-btrfs/snapshots/2022-09-01_09-55-22/@ | ondemand | {timeshift-autosnap} {created before upgrade} |
Found snapshot: 2022-08-29 12:26:16 | timeshift-btrfs/snapshots/2022-08-29_12-26-16/@ | ondemand | {timeshift-autosnap} {created before upgrade} |
Found snapshot: 2022-08-29 12:26:09 | timeshift-btrfs/snapshots/2022-08-29_12-26-09/@ | ondemand | {timeshift-autosnap} {created before upgrade} |
Found snapshot: 2022-08-29 12:24:31 | timeshift-btrfs/snapshots/2022-08-29_12-24-31/@ | ondemand | {timeshift-autosnap} {created before upgrade} |
Found snapshot: 2022-08-29 09:55:09 | timeshift-btrfs/snapshots/2022-08-29_09-55-09/@ | ondemand | {timeshift-autosnap} {created before upgrade} |
Found snapshot: 2022-08-29 09:49:08 | timeshift-btrfs/snapshots/2022-08-29_09-49-08/@ | ondemand | {timeshift-autosnap} {created before upgrade} |
Found snapshot: 2022-07-29 14:23:29 | timeshift-btrfs/snapshots/2022-08-29_12-05-01/@ | ondemand | Before restoring '2022-08-23 10:06:44'        |
Found 7 snapshot(s)
Unmount /tmp/grub-btrfs.J65h9x3QUM .. Success
done
...

@Schievel1
Copy link
Collaborator

I am working on a daemon right now to watch for snapshots and triggering grub-btrsd instead of relying on systemd mounts and its almost finished. Maybe you want to give it a try. I put instructions into the PR, #218, for more instructions visit https://github.com/Schievel1/grub-btrfs/tree/update-daemon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants