How does restic handle file changes while it is creating a snaphot?
Since my linux system is running on BTRFS, I would think it’s a good idea to take a temporary BTRFS snapshot before running restic backup and delete it afterwards.
Does restic provide any functionality to accomodate this?
Are there any best practices or bash snippets I could base my solution on?
Reading this once again, I think I should make this more precise:
I’d like to make a BTRFS snapshot and backup from this, but of course this has to be the original path from restic’s point of view (not some special BTRFS path that points to the snapshot and is interpreted as something completely unrelated in the restic snapshot list).
Restic does not offer something like this out of the box - for Windows there is a WSS functionality built in.
What you can do is use a Systemd Unit triggered by a Systemd Timer with a ExecStartPre= option which does the BTRFS snapshot for you.
That’s what I do for my systems and it works very reliably.
The only way I could think of is to use a chroot to change the directory for the backup.
So you would have to play around with chroots to test this out and get it working.
Apart from that, here’s what I have setup:
# restic.system.service
[Unit]
Description=[restic] Backing up system folders
[Service]
# Run all of the commands below as this user
User=root
Group=root
Type=oneshot
EnvironmentFile=/etc/restic/B2.env
EnvironmentFile=/etc/restic/repo.env
ExecStartPre=/bin/btrfs subvolume snapshot -r / /mnt/btrfs-snapshots/restic
ExecStartPre=/usr/local/bin/restic unlock
ExecStart=/bin/echo "[Script] Backing up system folders now"
ExecStart=/usr/local/bin/restic backup \
/mnt/btrfs-snapshots/restic/etc/ \
/mnt/btrfs-snapshots/restic/root/ \
/mnt/btrfs-snapshots/restic/usr/local/ \
/mnt/btrfs-snapshots/restic/var/ \
/mnt/btrfs-snapshots/restic/opt/ \
--exclude /mnt/btrfs-snapshots/restic/var/backups/ \
--exclude /mnt/btrfs-snapshots/restic/var/cache/ \
--exclude /mnt/btrfs-snapshots/restic/var/lock/ \
--exclude /mnt/btrfs-snapshots/restic/var/run/ \
--exclude /mnt/btrfs-snapshots/restic/var/spool/ \
--exclude /mnt/btrfs-snapshots/restic/var/tmp/ \
--exclude /mnt/btrfs-snapshots/restic/var/lib/ \
--exclude /mnt/btrfs-snapshots/restic/var/log/ \
--exclude-caches \
--one-file-system \
--cache-dir /var/lib/restic/cache/ \
--tag 'systemd system folders' \
-o b2.connections=30
ExecStart=/bin/echo "[Script] Done. No more tasks to do..."
ExecStartPost=/bin/btrfs subvolume delete /mnt/btrfs-snapshots/restic
[Install]
WantedBy=multi-user.target
# restic.system.timer
[Unit]
Description=This is a daily timer for a restic system backup.
[Timer]
OnCalendar=*-*-* 02:00:00
Unit=restic.system.service
Persistent=true
[Install]
WantedBy=basic.target
Note: There is no checking to see if a BTRFS Snapshot exists or do a prior deletion of the snapshot in case there is one. You can also build logic to do snapshots in a path with a unix epoch timestamp so you can be sure that if the backup did not finish last time, you still have the snapshot around.
Note 2: Some may say that passing secrets via process environments is a security risk. So be the judge of that how you pass the restic repo and passphrase. There is also the --password-file and --repository-file which, now that I’m writing this, is something I will switch to.
If anyone else is reading this and has thoughts, don’t be shy 
This looks very promising - thank you very much for sharing the config of your systemd unit!
I am not very familiar with this, but based on such a template, I will definitely give it a try and find out how it works
Using a --password-file with 600 privileges is something I practice already and can only recommend.
I don’t really see the advantage of putting the repo URL into a file instead of a command line parameter - at least not for my use case.
This method is working beautifully
I changed the structure of the scripts according to my needs and put most of the restic stuff into a bash script file, that now is the core of the service.
I am totally baffled by how fast this whole procedure is working!
One little thing I am not entirely happy about:
Now the path of the snapshot in the restic repo contains the BTRFS snapshot mount point instead of the original path.
Does restic have an option to configure a path alias?
Example: I make a backup of /mnt/restic/btrfs-snapshot/home and tell restic that it should consider this to be /home, so this snapshot will look exactly like one taken without the BTRFS snapshot.
I couldn’t find this in the help, but before giving up on this, I thought I’d better ask people who really know this software…
Awesome that you’re happt with the solution.
So the only way I could see to have a different base path, let’s call it a prefix, for your system is to use a chroot. Check the forum Search results for 'chroot' - restic forum to see if others have already posted something about this. As you’re definitely not the only one asking for this.
Let us know how it went!
Using chroot sounds like a good idea, but it seems to have it’s pitfalls.
Taking a btrfs snapshot of /home is not sufficient, because then I cannot successfully chroot into the parent folder, because of the missing system environment.
So I first need to make a snapshot of /, which cannot be readonly, because I have to remove the home folder inside the snapshot before taking a snapshot of /home at the right spot.
Then I can chroot into the appropriate directory.
Here are the commands I am using (the empty folder /mnt/test exists):
btrfs subvolume snapshot / /mnt/test/snapshot
rmdir /mnt/test/snapshot/home/
btrfs subvolume snapshot /home /mnt/test/snapshot/home
chroot /mnt/test/snapshot/ bash
But this doesn’t really work - the chroot is done, but with every command I get a long list of python errors that all boil down to KeyError: (('/proc',), frozenset())
This might only be some small piece of configuration, I am missing - but I have no idea what it could be.
I’d appreciate hints from people who have an understanding of chroot, wich I am lacking.
I just took another look at the error messages and it seems, the chroot environment is missing /proc/stat:
FileNotFoundError: [Errno 2] No such file or directory: '/proc/stat'
So I tried to solve this with a symlink:
ln -s /proc/stat /mnt/test/snapshot/proc/stat
But this only changes the error message to
OSError: [Errno 40] Too many levels of symbolic links: '/proc/stat'
Now I am running out of ideas…
You have to use mount -t proc proc /mnt/test/snapshot/proc or similar. Symlinks cannot leave a chroot, instead they are interpreted from within the chroot (otherwise the chroot would be useless for security purposes). That is once you’re inside the chroot then /proc/stat (which from the outside is /mnt/test/snapshot/proc/stat) points to /proc/stat and therefore is a symlink pointing to itself.
Yes!
Using mount instead of ln works (with exactly the command you posted)!
Now I can chroot into my snapshot and even use restic there. 
The only remaining problem is that network name resolution does not work inside the chroot environment. I can work with my rest-server using it’s IP, but the host name cannot be resolved.
I assume that’s another feature of chroot - is there a simple way to solve this inconvenience?
What is the content of your /etc/resolv.conf? I guess it’s something that depends on e.g. /run . In that case you can use mount --bind /run /mnt/.../run to make that folder accessible from within the chroot.
Thank you very much, Michael!
Once again you were right and mounting /run inside the chroot environment solved the name resolution issue.
In my /etc/resolv.conf I don’t see any mention of /run, but I guess that might be implicit in the line “nameserver 127.0.0.53”.
I am very happy with the way my restic backup systemd service unit is working now thanks to the great support I got in this friendly forum
One could say it was too much effort to get some rather “cosmetic” aspects right, but I learned many new things about systemd, btrfs and chroot - so for me it was totally worth the time 
Thanks to all the people that make this great tool even better by providing such valuable help for real world use cases!
That sounds a lot like /etc/resolv.conf is a symlink pointing to somewhere in /run/.
That’s exactly what I missed
@silmaril42 Maybe you can share your setup with us so others coming from Google or the forum in general can integrate it themselves That would be really nice
Good idea
That’s a very good opportunity to clean up the scripts and document the most important aspects.
Comments and suggestions welcome
Since backups are performed regularly, and restores less often, how about symlinking the snapshot prefixes before restoring then cleaning up?
eg
ln -s /home /mnt/snapshots/home
restic restore ...
rm /mnt/snapshots/home