What are the impacts of a large number of snapshots in the repository? I’ve found some topics, but they are more than 4 years old and restic has developed a lot since then.
Suppose we have 2 repository with ~2 TB and an average file size of 5 MB and 1000 snapshots.
How much more RAM is needed for backup command when there are 1000 snapshots instead of only 20?
What impacts are on the forget and prune command? It could be possible that there are almost no impacts for backup, but prune and forget might need so much memory that you need extra hardware for removing old snapshots.
I am not sure about RAM but definitely you might need 100s of GB space for local cache. restic at the moment does not have any mechanism to limit its size. It only depends on repo data. You find many threads on this forum where people can’t run prune as they do not have enough disk space for cache e.g. Massive cache size? - #7 by CendioOssman.
There is no simple formula to estimate its size - but more snapshots and more data changes bigger cache becomes.
Thanks for the Info.
Thats very interesting. For 1480 snapshots with a repository size of 3 TB I had about 451 MB cache size. After trying to get some stats via stats command cache increased to 1,5 GB, then I aborted restic command and deleted the files created a few minutes ago from cache again.
Another repo with 8.8 TB size and about 20 snapshots uses ~280 MB for cache.
So when just doing backup the cachesize is no problem. And pruning can be done without cache, which is no problem, since the files reside in LAN or maybe even on the machine restic is running.
Hope there will be some answers regarding RAM usage and number of snapshots.
The number of snapshot only has a minor influence on the memory usage of the backup command. What’s much more important is the size of the repository index, as it has to be loaded for all operations. There are two notable exceptions to this rule: folder with more than 100k files or that contain more than 1TB (directly in the folder, subfolders are not a problem).
Prune could require about 2-3 times the memory of the backup command. There, the main memory usage also originates from the index.