Possible way to speed up scans on Windows?

Features and Ideas
1

I’ve been trying out restic and Duplicacy this weekend, and one thing I noticed is that restic is much slower when it comes to scanning for changes. Initially the problem was due to Window’s built in antivirus protection, but even with that disabled scanning is slow in comparison.

Using Process Monitor I noticed that running Duplicacy on a directory containing about 100,000 files resulted in 35,260 events in Process Monitor, most of them of the type IRP_MJ_DIRECTORY_CONTROL on directories, requesting info about 30 files per operation. restic on the other hand generated 984,668 events (almost 30 times as many) for the same files, with operations on every single file.

I don’t know Go and have no knowledge of restic’s source code so I apologize if this doesn’t make any sense, but would it be possible to speed up scans by changing the way it looks for changes, similar to how Duplicacy does it?

2

AFAIK fd0, the author of restic, doesn’t use Windows himself. So the only reason (big *) restic is available on Windows is due to the fact that Go can be compiled for so many different architectures and OSs easily.
If you look through the Github project there have been some improvements to issues on Windows but it is not on the radar in that regard - if I might say.

So the fact that restic isn’t optimized to its full potential for Windows is very likely.

3

I’m sure contributions would be welcomed in this regard! (I’m also not a Windows user myself.)

4

Oh yes, definitely. I did not mean it in a discouraging way at all :slight_smile:

1 Like
5

Restric should make use of multi core / threads depending on availability of system resources.
& Or
To speed up scanning on windows NTFS file systems make use of USN journal to pick up only changed & new files.

6

Not sure if this is relevant to you, but make sure you always use parent snapshots when backing up with restic, otherwise it has to (re)read all the files from disk on each backup.

You can tell if restic is using a parent snapshot by looking in the backup output for a line such as “using parent snapshot d875ae93”. The default is to use the “last snapshot in the repo that has the same target files/directories”.

If that doesn’t work in your case, you can specify the snapshot manually using the --parent string argument.