I’m testing restic 0.9.4 as a possible solution for backing up relatively large collection of files (a server users’ home directories and research data files) to a remote storage via sftp backend. The data size of both, the remote restic repository and the original collection, is about 250GB. There is ~50K files in the repository and ~780K in the original collection. Both, my server and the remote storage site, run CentOS 7.
I’m fully satisfied with the backing up process (really decent performance!). However, restoring data raises questions. In my first test I tried to restore a 3.3GB, ~400 files (sub-)directory tree (a small subset of the original collection):
restic -r sftp:remote_host:path_to_repo restore latest --include path_to_subtree --verbose --target /scratch/
it took 24min, thus, the average data transfer rate was only 2.2MB/s. The network link performance between the local server and the remote storage site is 30-40MB/s (experience of routinely using it) and the test was repeated multiple times. So, 2.2MB/s is not a random drop in the link performance.
In the following up tests I moved the repository around: put it on another local server and 2 more remote sites located at different distances (so network latency varied from 0.05ms to 50ms). Then, I restored the same subtree on the same server with the command above just varying the remote repository and cleaning up /scratch directory before each test. Here is my results:
latency to the test (restore) Avg.data transf. repository host time rate 0.05ms(local host) 53s 60MB/s 1ms 130s 25MB/s 50ms(orig.test) 1450s 2.2MB/s 50ms(anoth.site) 1490s 2.2MB/s
Each test was repeated multiple times to ensure that the results are reproduced. So, it looks like it’s a feature of restic to slowdown with the distance to the repository location (in terms of network lag). Is that right ? Or should I look for other reasons of the poor restic restore performance for the far (50ms) sites ?