Surprisingly slow incremental backups


#1

I did my first backup with restic during the night, about 300 GB in ~100,000 files. I then ran the same command again to make a new snapshot with just a few new files added, and to my surprise it took 40 minutes. By comparison the same incremental backup in duplicacy took a mere 18 seconds.

Below is the command and output of the second snapshot with just a few new files:

restic -r rclone:gdrive:restic backup --verbose --exclude-if-present ".nobackup" --exclude-file "D:\restic filters.txt" --limit-upload 8500 --tag docs D:\Documents
open repository
enter password for repository:
repository 1f6315ed opened successfully, password is correct
lock repository
load index files
using parent snapshot 3cefb80c
start scan on [D:\Documents]
start backup on [D:\Documents]
scan finished in 17.755s: 110609 files, 291.752 GiB
uploaded intermediate index b9d3dc55

Files:          25 new,     3 changed, 110582 unmodified
Dirs:            0 new,     1 changed,     0 unmodified
Data Blobs:     39 new
Tree Blobs:      2 new
Added to the repo: 30.735 MiB

processed 110610 files, 291.752 GiB in 40:31
snapshot fe485469 saved

Afterwards I ran the same command once again, but cancelled it since the estimated time was about the same.

[20:06] 49.33%  53441 files 143.924 GiB, total 110616 files 291.765 GiB, 0 errors ETA 20:38

Am I right in thinking incremental snapshots should not take this much time?

I’m using restic version 0.9.4.


#2

Judging by your command line your are using a Windows client. Most probably your antivirus software is slowing down restic. Please see this post and this post for more information.


#3

Thank you, that was it! I’ve never had a problem with Windows Defender before (that I know of), but in this case it was definitely the culprit. After excluding the restic process, scanning now takes two minutes instead of 40. Still not as quick as duplicacy’s 18 seconds, but two minutes I can live with.