Looking to improve backup speed for large folder

Hi, thanks for developing restic, is my main backup tool!

I made a script to automate the backup of my docker folder from my NAS to my local workstation on a spare disk.
The biggest folder on my NAS is the immich library that it takes around ~260gb of pictures.
The first time I run restic it takes a lot of time to create the backup and this is expected.
But even if I do not change to much files, the next backup is taking the same time. I’m wondering if there is a way to backup only the new files.

Here some details:

➜  ~ restic version 
restic 0.17.1 compiled with go1.23.1 on linux/amd64

In my script, I mount the remote snapshot on a local mount point using this command:

sshfs user@192.168.10.1:/docker/#snapshot/GMT+02-2024.10.17-01.00.02 /mnt/sshfs/nas/docker

This is an excerpt of my script with restic details:

# Restic configuration
exclude="$HOME/.config/backup/exclude_patterns"
restic_external_repo="$external_disk/restic/nas-$folder_to_backup"
restic_local_repo="/volumes/restic/nas-$folder_to_backup"
restic_passwd="$HOME/.password/restic-credentials"
keep_options="--keep-daily 6 --keep-weekly 3 --keep-monthly 1"

The restic commands I run in the script:

restic -p $restic_passwd -r $restic_local_repo unlock
restic -p $restic_passwd -r $restic_local_repo backup --exclude-file $exclude $snapshot_local_mnt || echo "Restic backup failed"
restic -p $restic_passwd -r $restic_local_repo forget $keep_options --prune --cleanup-cache

umount $snapshot_local_mnt

Some details:

  • the synology nas is configured using BTRFS and takes daily snapshots of the docker folder;
  • I do the backup of the snapshot to be sure it is not going to change during the backup itself; I think it’s a best practice but happy to be contradicted.

These is and excerpt of the restic output:

Snapshot is mounted and restic folder exists...
restic: backup of /mnt/sshfs/nas/docker on local repo /volumes/restic/nas-docker

repository 2da9119a opened (version 2, compression level auto)
repository 2da9119a opened (version 2, compression level auto)
using parent snapshot e977cda4
[0:00] 100.00%  11 / 11 index files loaded
[23:23] 8994 files 52.624 GiB, total 48122 files 207.859 GiB, 0 errors

I would like to see if there is anything I can do to speed up the next backups.

Thanks!

If I remember correctly, sshs does not ensure stable inodes. That is the backup command assumes that all files have changed on each backup. Try adding --ignore-inode to the backup command.

1 Like

Thanks @MichaelEischer so far no issues with this option! But I’ll keep an eye.