Hello,
I’m testing restic for the first time 
I made no changes to my files yet backuping again is very slow 1h for 130go / 12k files.
Source folder is on an external HD USB3
Repo is on another an external HD USB2
In between I use ubuntu.
nicolas@laptop ~ » restic -r /media/nicolas/Backups/nicolas/images/2017 --verbose backup /media/nicolas/Images/2017
open repository
enter password for repository:
repository 73d282e0 opened successfully, password is correct
lock repository
load index files
using parent snapshot b0bbf794
start scan on [/media/nicolas/Images/2017]
start backup on [/media/nicolas/Images/2017]
scan finished in 41.629s: 12227 files, 131.953 GiB
uploaded intermediate index 240dbf48
Files: 0 new, 12227 changed, 0 unmodified
Dirs: 0 new, 3 changed, 0 unmodified
Data Blobs: 0 new
Tree Blobs: 4 new
Added to the repo: 1.530 KiB
processed 12227 files, 131.953 GiB in 56:51
snapshot 6db79888 saved
If I make a diff between the two last snapshots :
nicolas@laptop ~ » restic -r /media/nicolas/Backups/nicolas/images/2017 diff b0bbf794 6db79888
enter password for repository:
repository 73d282e0 opened successfully, password is correct
comparing snapshot b0bbf794 to 6db79888:
Files: 0 new, 0 removed, 0 changed
Dirs: 0 new, 0 removed
Others: 0 new, 0 removed
Data Blobs: 0 new, 0 removed
Tree Blobs: 251 new, 251 removed
Added: 10.164 MiB
Removed: 10.164 MiB
Is this the expected behaviour ?
Thank you for your help !
edit:
Restic version :
restic 0.9.6 compiled with go1.14.7 on linux/amd64
1 Like
It depends on the read/write speeds of your drives and their caching settings and the drive specification, storage type (SMR or not), and what other I/O the drives and system are under. Newer versions of restic are more performant, but you haven’t indicated the output from restic version.
Hello,
I’m using version 0.9.6 compiled with go1.14.7 on linux/amd64.
I’m updating to 0.12 and trying again.
1 Like
I ran the same operation after updating, and the result is the same:
nicolas@laptop ~ » restic version
restic 0.12.0 compiled with go1.15.8 on linux/amd64
nicolas@laptop ~ » restic -r /media/nicolas/Backups/nicolas/images/2017 --verbose backup /media/nicolas/Images/2017
open repository
enter password for repository:
repository 73d282e0 opened successfully, password is correct
lock repository
load index files
using parent snapshot 6db79888
start scan on [/media/nicolas/Images/2017]
start backup on [/media/nicolas/Images/2017]
scan finished in 1.500s: 12227 files, 131.953 GiB
Files: 0 new, 12227 changed, 0 unmodified
Dirs: 0 new, 297 changed, 0 unmodified
Data Blobs: 0 new
Tree Blobs: 252 new
Added to the repo: 10.164 MiB
processed 12227 files, 131.953 GiB in 55:33
snapshot 6be53f81 saved
I haven’t changed the files on the disk, but still 12227 files with the status changed and 1 hour of processing.
On which filesystem is the source folder stored? It looks like some metadata of these files is different every time and thus restic has to reread all files.
nicolas@laptop ~ » df -Th | grep "^/dev"
/dev/sda6 ---redacted---
/dev/sda5 ---redacted---
/dev/sdb1 exfat 1.9T 696G 1.2T 38% /media/nicolas/Images
/dev/sdc1 ext4 3.6T 568G 2.9T 17% /media/nicolas/Backups
source’s disk is exfat
repo’s disk is ext4
So, thanks to your hint I’ve found this issue: Restic fails to recognize that files didn't change after mount cycle on fat32 · Issue #2058 · restic/restic · GitHub
I compared stat before and after re-mounting and indeed inode value is changing.
So I used --ignore-inode and it was fast (40s), and now it looks like the expected behaviour
nicolas@laptop ~ » restic -r /media/nicolas/Backups/nicolas/images/2017 --verbose backup /media/nicolas/Images/2017 --ignore-inode
open repository
enter password for repository:
repository 73d282e0 opened successfully, password is correct
lock repository
load index files
using parent snapshot 6be53f81
start scan on [/media/nicolas/Images/2017]
start backup on [/media/nicolas/Images/2017]
scan finished in 37.834s: 12227 files, 131.953 GiB
Files: 0 new, 0 changed, 12227 unmodified
Dirs: 0 new, 297 changed, 0 unmodified
Data Blobs: 0 new
Tree Blobs: 252 new
Added to the repo: 10.164 MiB
processed 12227 files, 131.953 GiB in 0:38
snapshot 0427b7fc saved
Thanks a lot.
1 Like