Backing up from rclone mount from btrfs snapshots


my setup was that I have a remote server from which I want to backup certain directories to my local USB drive. Previously, I used rsync to copy the files, create a new btrfs snapshot and delete snapshots that are older than a certain amount of days.

However, I like restic more (all other backups of my systems are with restic), so I want to migrate.
I think I already migrated the snapshots (with the --time setting for the correct dates and the --parent to select the correct parent). The repo path is /media/backup-drive.

The snapshots are listed correctly, however, with their full path in the path column:

$ restic snapshots
ID        Time                 Paths
9988ade4  2023-01-01 00:00:00  /media/backup-drive/_SNAPSHOTS/2023-01-01
8ed49944  2023-02-01 00:00:00  /media/backup-drive/_SNAPSHOTS/2023-02-01
47280a40  2023-03-01 00:00:00  /media/backup-drive/_SNAPSHOTS/2023-03-01
f392a1d7  2023-04-01 00:00:00  /media/backup-drive/_SNAPSHOTS/2023-04-01
29e55e7a  2023-05-01 00:00:00  /media/backup-drive/_CURRENT
18 snapshots

I believe this works as intended because the later snapshots were much quicker and I could see that only modified files were copied. These directories contained only the folders home/myremoteuser and data/myremoteuser.
I have create the snapshots inside the respective folder like this:

cd /media/backup-drive/_SNAPSHOTS/2023-02-01
restic backup --parent 9988ade4 --time="2023-02-01 00:00:00" .

Now, for the first real backup, I have mounted the remote via rclone mount remote:/ /remote and I’m running the backup with:

cd /remote
restic backup \
    --parent 29e55e7a \
    --exclude-file=excludes.txt \
    --files-from includes.txt

The excludes.txt contains only **/*.py[cd], and the includes.txt my two directories:


So, my question is two-fold: First, is this approach the best-practice? Do I benefit from incremental updates with this?

And second, I’m doubting it works correctly because it is reading all files from the remote which it doesn’t need to because I know that there were not many changes between now and the last snapshot.
The dry-run tells me that I’ll take around 6 hours for the backup.

When looking at the output of the --verbose=2 setting, I can see that many files are listed as modified, however, I know that I didn’t touch these files, and I tells me that with
modified /home/myremoteuser/arandomfile saved in 0.017s (0 B added, 0 B stored).

Could this be a remnant of the btrfs inode strategy and subsequent backups will be much faster or am I doing this wrong?

Additionally, with rsync, it only copied the files that were newer than locally. Does this work with a local rclone mount via SFTP? If it has to transfer all files (~1 TB) for each backup, I guess that this might not be the best backup strategy?

$ restic version
restic 0.15.2 compiled with go1.20.3 on linux/amd64

Probably the same as here:

This is old problem - and not solved yet.

Okay, thanks, so this suggests that it’s doing this once but not for the following backups?

backup from snapshots always treats all data as changed. So will read everything every time. Now because of deduplication no new data will be backed up. Only some metadata.

If you can live with it than all is fine. Problem is when you backup big amount of data - and re scanning everything takes long time.

Also if you have a lot of files (millions) every new snapshot metadata can be quite big when in reality nothing is changing.

I gave it the parent snapshot and judging from the speed of creating the snapshots in this 1TB repository, I assume that it actually didn’t read everything over and over.

That is only true if the snapshots use different paths. backup always has to fully read files if their path has changed. That happens independently of whether snapshots are used or not.

Specifying a parent snapshot only helps if there’s some overlap between the filenames. Deduplication of file contents always works independent of their filename or whether a parent snapshot was specified.

yes sorry i cut corners with my explanation. Maybe restic 17 or 18 can take snapshot world into account finally.

Anyway - it is great software. It is not perfect. Nothing is. Thank you for providing it.