I’m running restic backup --files-from /home/daniel/.config/restic/include/monthly --tag monthly which according to restic, has an ETA of about 6 hours. Given that I do backups daily, weekly, and monthly of exactly the same data (the monthly has the least changes and includes the daily and weekly changes), it’s perplexing to me that it will take as long as 6 hours. The data is mostly the same every backup.
The expectation is that doing two backups back to back with the same data would be rather fast, is it not?
It’s not my intention to withhold info when asking for help. I’m not sure what else I can provide aside from the command and the other details I already mentioned. What else would be of use?
[~/.c/restic]$ make monthly
building includes...
restic backup --files-from /home/daniel/.config/restic/include/monthly --tag monthly
repository 3a6e47e7 opened (version 1)
no parent snapshot found, will read all files
[0:10] 100.00% 68 / 68 index files loaded
Files: 248331 new, 0 changed, 0 unmodified
Dirs: 20575 new, 0 changed, 0 unmodified
Added to the repository: 449.850 MiB (450.301 MiB stored)
processed 248331 files, 1.054 TiB in 5:52:00
snapshot 50e8cf60 saved
it took 6 hours and 52 minutes to finish.
As mentioned, I run this command once per month. I backup a subset of the files weekly, and daily.
OK, I re-ran it again, and now it took 12 minutes. I am not sure what happened. I did have to rebuild my drive recently, but the files should have had the same hash values.
The first time you back up a given set of paths, restic has to scan all of the files. On subsequent runs of the same set of paths, it should be able to identify the parent snapshot that was previously created, and only scan the files that have any indication of possibly having been changed. You can see this by restic saying “using parent snapshot …” in the output. If this is not the case, it hasn’t found a parent snapshot to compare the list of files with, and hence need to re-scan all the files during the backup run.
Explore using incremental or differential backup methods to only process and back up new or modified files instead of the entire dataset each time and no other resource-intensive tasks are running simultaneously impacting the backup process.
Review Restic documentation for potential command optimizations or flags that might improve backup performance for recurring backups of mostly unchanged data.