New to restic, basic question: should it rescan all files everytime?

My apologies, I didn’t mean to come across rude but yes, re-reading my reply I agree with you.
So consider that I have slapped myself on the head for being rude and again, my apologies.
In my (pathetic) defence, I’d missed the second post where you put in a lot of details.

The version is pretty key because there’s often many bugs/things fixed with newer versions obviously, yet people often pop up running very old versions. I would try with 0.17.2 just to see if the behaviour changes, though I don’t think looking at the changelog that there’s anything in there that would affect rescan.

A few more questions so that someone with more clue than me can reply:
Is the External Drive just standard ext4? It’s not mounted with any strange mount flags, is it?
Is the restic cache still available, you haven’t deleted it?

When restic runs it has to rescan all the files to check for modifications or not, but I can’t fathom why that would take the same amount of time. Have you left it to finish the full scan and it took the same amount of time, or are you just concerned it’s doing a full-rescan and are stopping it?

For example on my box that’s got 15G of data on it:

-!- ~ » restic backup -v -r s3:<remote> --exclude-file=/etc/restic.exclude --exclude-caches --exclude-if-present .noresticbackup /var /usr/local /etc /home /boot /root
open repository
repository eac59ab7 opened (version 2, compression level max)
using parent snapshot a355f7df
load index files
[0:03] 100.00%  267 / 267 index files loaded
start scan on [/var /usr/local /etc /home /boot /root]
start backup on [/var /usr/local /etc /home /boot /root]
scan finished in 12.692s: 223392 files, 15.271 GiB. <---- FULL SCAN BUT IN 12s

Files:          55 new,    71 changed, 223266 unmodified
Dirs:            0 new,    67 changed,  2557 unmodified
Data Blobs:    186 new
Tree Blobs:     52 new
Added to the repository: 101.057 MiB (21.747 MiB stored)

processed 223392 files, 15.271 GiB in 0:38
snapshot c6184166 saved

So yea, it’s normal for all files to be “scanned” but it’s just checking to see if the file has been modified or not and it does 15G in very short amount of time. I expect an external USB would be orders of magnitude slower than that.

2 Likes