Recurring Backup with restic 0.9.5 takes like forever

Getting Help
1

Hi,

I just upgraded from restic 0.8.3 to 0.9.5.
I’m running Restic on a Debian Stretch host.

No matter whether I’m doing a recurring full system backup (with several file exclusions) or a backup of my Nextcloud over WebDAV, the backup (backup target is a SFTP server within the same datacenter, Gigabit connection) takes ways longer with 0.9.5 than it did with 0.8.3

Full System Backup (around 100 GB, 186339 files):
0.8.3: 8 minutes
0.9.5: 2 hours 50 minutes

Nextcloud Backup (WebDAV/davfs2, 62 GB, 14543 files):
0.8.3: <2 minutes
0.9.5: 4 hours 56 minutes

Any ideas whether restic changed the way how it discovers files? (I just did a brief look into the changelog but didn’t find any eye-catching change)
Any idea how I can trace down which component is responsible for that?

Call to restic is
Fullbackup:
restic -r sftp://storageHost/path --password-file pwfile backup --exclude-file=excludefile --one-file-system /
Nextcloud backup:
restic -r sftp://storageHost/path --password-file pwfile backup --exclude-file=excludefile .

The exclude files have less than 30 lines.

2

Quoting from the 0.9.0 release notes:

  • Enhancement #549: Rework archiver code
    The core archiver code and the complementary code for the backup command was rewritten completely. This resolves very annoying issues such as 549. The first backup with this release of restic will likely result in all files being re-read locally, so it will take a lot longer. The next backup after that will be fast again.
    […]
1 Like
3

Thanks, seems that I overlooked that.

Unfortunately even after the 3rd run following the upgrade, the speed didn’t improve:

Files:       14426 new,     0 changed,   150 unmodified
Dirs:            0 new,     0 changed,     0 unmodified
Added to the repo: 409.075 MiB

processed 14576 files, 61.550 GiB in 3:05:32
snapshot a4acf047 saved
4

Did you make sure that your backups aren’t limited by upload speed? Any other uploads happening at the moment?

5

There are no other uploads at the same time and both parties are servers in a data center with a 1Gbit/s pipe.

6

Where is that output from? Full System Backup or Nextcloud Backup?

I suggest you a backup with --verbose mode enabled.

7

Thanks.

Did that.

That’s the response:

`open repository
lock repository
load index files
using parent snapshot ddbcb2d5
start scan on [.]
start backup on [.]
scan finished in 2344.020s: 15422 files, 66.330 GiB

Files: 17 new, 0 changed, 15405 unmodified
Dirs: 0 new, 0 changed, 0 unmodified
Data Blobs: 59 new
Tree Blobs: 1 new
Added to the repo: 62.573 MiB

processed 15422 files, 66.330 GiB in 4:38:11
snapshot 12c44ca6 saved
`

Edit: And it’s the Nextcloud Backup. - Before the upgrade of restic it took no more than 2 minutes.

8

39 minutes is way too long to scan 15k files. On 1 of my systems with spinning disks and restic 0.9.5 it takes 18 seconds to scan 176k files. Maybe you can do some benchmarking to see if this delay is caused by hardware.