Restic backup is slow and fetches a huge amount of data in the same time

Getting Help
1

Using Restic 0.12.1 on Debian. The backup is sent via sftp to a server with gigabit network on the route.

Most of the data should already be on the target backup system, since the last backup was not very long ago. The screenshot shows the backup source on the left and the backup destination system on the right. Right. On the left the source, on the right the destination. As you can see here, the traffic goes from the destination to the source. It feels like the whole repository is fetched from resitc.

Bildschirmfoto von 2021-12-09 23-35-03
Bildschirmfoto von 2021-12-09 23-35-031499Ă—427 115 KB

I have been using restic for a very long time and I have a feeling that it was faster in older versions as well, which unfortunately I cannot determine. But that the whole repository has to be transferred over the net for a backup to the source system seems like a bug to me.

2

Please paste the complete command you use the run the backup, and all of its output.

3

Hi @rawtaz,

please find below the requested informations.

time RESTIC_REPOSITORY=sftp:johndoe@10.0.0.10:/restic RESTIC_PASSWORD=secret restic backup --exclude-caches --exclude-file exclude --tag "johndoe" --files-from include

repository 594tja21 opened successfully, password is correct
using parent snapshot 6u5add28

Files:          70 new, 71725 changed, 148422 unmodified
Dirs:            5 new,  5399 changed,  3986 unmodified
Added to the repo: 168.182 MiB

processed 220217 files, 1.441 TiB in 2:32:34
snapshot 9d4ff32 saved

real	152m35,234s
user	197m49,052s
sys	12m44,825s
4

Thanks! As you can see in the output, restic only sent 168.182 MiB of data to the repository. Presumably that isn’t your entire repository :slight_smile:

It seems a lot of files changed though, is it realistic that there are 71725 changed files? If not this might be due to metadata changes and if so you might want to investigate if you need to use either of the --ignore-ctime or --ignore-inode options to the backup command.

Also note that if the file list in the include file changed since last time and isn’t identical to an existing snapshot’s path list, restic won’t be able to find a snapshot to use as the “parent” when scanning the files. In that case it will simply scan all the files again (but still only upload the actual blocks that changed in the files). This might take a bit longer than when it can find a parent snapshot (which you can set manually with the --parent option).

1 Like
5

His described problem is the other way around. They complain about the complete repository being downloaded.

6

The total runtime and the number of files just don’t add up. Based on the amount of data added to the repository the whole backup should take only a few minutes. What is odd is that one third of the files are detected as changed, which consequently causes restic to read those again. Based on the high network usage I’d assume that you’re backing up a network mount which somehow causes restic to think the files have changed.

You can use backup -vvv to let restic print which files were detected as changed. It might be necessary to add --ignore-ctime or --ignore-inode to the backup command.

7

Their “It feels like the whole repository is fetched from resitc” implies they meant from their restic client to the repository, but okay.

8

please find below a second run without --ignore-ctime or --ignore-inode. I will try it with these next time.

repository 432dea11 opened successfully, password is correct
using parent snapshot 9dff4516

Files:          89 new,  5589 changed, 142776 unmodified
Dirs:           12 new,   723 changed,  8430 unmodified
Added to the repo: 119.979 MiB

processed 148454 files, 721.116 GiB in 0:58
snapshot 3rr57e80 saved

real	0m59,253s
user	2m3,008s
sys	0m11,795s
9

Please find below a run with --ignore-inode. looks like the speed is back in the usual range again.
What is the negative implication of this option?

repository 7d2cba11 opened successfully, password is correct
using parent snapshot 7fg57e80

Files:          73 new,    28 changed, 148426 unmodified
Dirs:          130 new,    36 changed,  9129 unmodified
Added to the repo: 23.492 MiB

processed 148527 files, 721.121 GiB in 0:33
snapshot 634bb39b saved

real	0m34,638s
user	1m4,070s
sys	0m7,717s
10

Please find below a run with --ignore-ctime. looks also like the speed is back in the usual range again.

repository 794fgd11 opened successfully, password is correct
using parent snapshot 634cc39b

Files:           8 new,    24 changed, 148500 unmodified
Dirs:            2 new,    31 changed,  9264 unmodified
Added to the repo: 12.027 MiB

processed 148532 files, 721.121 GiB in 0:34
snapshot 9f10feda saved

real	0m35,603s
user	1m4,694s
sys	0m8,179s
11

These backup runs don’t exhibit the initial problem any more. So there’s also no need for the additional options.

However, the set of files backed up has changed drastically, now the output is:

processed 148454 files, 721.116 GiB in 0:58

While initially it was:

processed 220217 files, 1.441 TiB in 2:32:34

So, whatever has changed between the backup runs, that change is the cause of the performance problems.

12

The request can be deleted. My mistake is solved with -x, sorry