Unclear restic cache behavior

Hello,

I’m running restic 0.14 on a raspberry pi backing up 25MB in 3400 files. The filesystem is tmpfs without swapping, so everything is in RAM.

I’m seeing a strange behavior and I’m not sure if this is expected.

  • when i run a backup job with --no-cache (even multiple repeat runs), restic appears to scan every single file and takes ~ > 20 seconds
  • when i run a backup job without no-cache, even though I just cleared the cache and restic has to rebuild it from scratch (as confirmed by created new cache in /run/restic), it is very fast and finishes in 2 seconds

Why is restic less performant with --no-cache even compared to an empty cache it has yet to rebuild?

Is it doing something dangerous in the empty cache situation?
Or is --no-cache just unnecessarily complex/slow for the lack of optimization?

Otherwise, caching into another tmpfs like /run/ and deleting the files afterwards would improve the situation without affecting ram or sd card after the backup job is done:

restic --cache-dir /run/restic/ backup /data/
rm -rf /run/restic/

Here’s the full output:

root@hauspi:~# restic version
restic 0.14.0 compiled with go1.19 on linux/arm
root@hauspi:~#
root@hauspi:~# restic --no-cache backup /pisrv/
repository cf8c7b03 opened (repository version 2) successfully, password is correct
using parent snapshot ac3805f7

Files:           0 new,     0 changed,  3415 unmodified
Dirs:            0 new,     0 changed,   399 unmodified
Added to the repository: 0 B   (0 B   stored)

processed 3415 files, 23.018 MiB in 0:25
snapshot 444d0a19 saved
root@hauspi:~#
root@hauspi:~# restic --no-cache backup /pisrv/
repository cf8c7b03 opened (repository version 2) successfully, password is correct
using parent snapshot 444d0a19

Files:           0 new,     0 changed,  3415 unmodified
Dirs:            0 new,     0 changed,   399 unmodified
Added to the repository: 0 B   (0 B   stored)

processed 3415 files, 23.018 MiB in 0:24
snapshot 9ac2d786 saved
root@hauspi:~#
root@hauspi:~# restic --no-cache backup /pisrv/
repository cf8c7b03 opened (repository version 2) successfully, password is correct
using parent snapshot 9ac2d786

Files:           0 new,     0 changed,  3415 unmodified
Dirs:            0 new,     0 changed,   399 unmodified
Added to the repository: 0 B   (0 B   stored)

processed 3415 files, 23.018 MiB in 0:25
snapshot bffe59af saved
root@hauspi:~#
root@hauspi:~#
root@hauspi:~# restic backup /pisrv/
repository cf8c7b03 opened (repository version 2) successfully, password is correct
created new cache in /run/restic
using parent snapshot bffe59af

Files:           0 new,     0 changed,  3415 unmodified
Dirs:            0 new,     0 changed,   399 unmodified
Added to the repository: 0 B   (0 B   stored)

processed 3415 files, 23.018 MiB in 0:02
snapshot 0103e5cb saved
root@hauspi:~#
root@hauspi:~# rm -rf /run/restic
root@hauspi:~#
root@hauspi:~# restic backup /pisrv/
repository cf8c7b03 opened (repository version 2) successfully, password is correct
created new cache in /run/restic
using parent snapshot 0103e5cb

Files:           0 new,     0 changed,  3415 unmodified
Dirs:            0 new,     0 changed,   399 unmodified
Added to the repository: 0 B   (0 B   stored)

processed 3415 files, 23.018 MiB in 0:02
snapshot 44546f34 saved
root@hauspi:~#
root@hauspi:~# rm -rf /run/restic
root@hauspi:~#
root@hauspi:~# restic backup /pisrv/
repository cf8c7b03 opened (repository version 2) successfully, password is correct
created new cache in /run/restic
using parent snapshot 44546f34

Files:           0 new,     0 changed,  3415 unmodified
Dirs:            0 new,     0 changed,   399 unmodified
Added to the repository: 0 B   (0 B   stored)

processed 3415 files, 23.018 MiB in 0:02
snapshot a9a0ca02 saved
root@hauspi:~#
root@hauspi:~#

In order for restic to check whether files have been modified since the last snapshot, it has to download the directory metadata from that snapshot. Without a cache this happens one by one for each directory which is pretty slow. With a cache, restic can just download the corresponding pack file (which contains the metadata for many directories) and then read all directory metadata from the local cache.

That performance assessment assumes that the repository is not stored on a local drive, but somewhere else. Is that the case?

What is the reason to not store the cache on the respberry?

Got it, the point of the question is couldn’t restic do this in RAM?

Correct, its an AWS S3 41ms away.

SD card wear.

I will move the cache to a /run/ which is on tmpfs.

It’s just seems like a non-obvious behavior.

Sure, we could optimize a bit for that case (although I doubt that it is worth the effort, the use case seems to be relatively special). However, it is not too uncommon for the metadata cache to be larger than the available main memory. Thus, we can’t just keep everything in memory which increases the complexity.

Yeah, I believe it’s not worth the effort.

And repositories so small are rare so memory usage would definitely be a problem in most cases.

Perhaps we should improve the documentation, making it clear how important the cache is and that --no-cache can make a huge difference for backup and restore operations when repositories are a few milliseconds away.

I’ll see if I can come up with some proposals.

1 Like