Hmmm yeah, it’s definitely going a little slow for you then. I wonder if the Finland servers would be faster for you than Germany (or vice-versa if you already went with Finland)?
Geographically, Finland is closer and results from below servers also reflect that,
http://fsn1-speed.hetzner.com/
http://hel1-speed.hetzner.com/
I was trying to automate the backup using script, but it seems it is really hard when a fatal error occurs. It doesn’t let me continue the rest of the script!
Backups over SFTP should be much, much faster starting from restic 0.17.0.
The reported speed was tested with restic 0.16.2. I don’t know, maybe it’s a Hetzner issue.
Slow sftp upload performance in restic before 0.17.0 was a bug in restic Slow SFTP transfer rate since 0.12.1 · Issue #4209 · restic/restic · GitHub .
Thanks for link. Just tested and now the avg upload speed is 2.24MiBps. I guess it’s a Hetzner issue.
My experience after using my Hetzner storage box in Finland (from Sweden so I’m close) for a couple of days now. My connection speed is 500/500 and that’s what fast.com tells me.
As of now, I use
Backrest (docker on Linux: garethgeorge/backrest:latest)
Included in the container as of now:
backrest 1.6.1
restic 0.17.1
rclone 1.66.0-DEV
So this is using SFTP via rclone.
First initial backup, brand new repo
102.21 GiB in 35m40s.
Files Added 16216.
All media files.
Acceptable. 
Backing up the same source using Duplicati to OneDrive (using the native implementation takes about 1h 30m)
I’d be curious to see the native implementation of SFTP compared, and Rcllone+WebDAV, if you’re up to it?
This is for rclone/WebDAV
Files Added:
16252
Bytes Added:
102.91 GiB
Time:
34m36s
I can try native sftp later today.
For me, from fastest to slowest, it’s…
- Rclone+WebDAV
- Rclone+SFTP
- Native SFTP
@akrabu Using which restic version? How fast is each variant?
Shoot, I totally did a test earlier and must have forgotten to put it here lol
I would have been on 0.17.1 when I first tested. I’m on 0.17.3 now. All my scripts call restic self-update so it’s pretty rare for me to not be running the latest (only when I’ve found a bug that impacts me, really).
Here’s a comparison of just “restic diff --no-cache” with two ~65MiB snapshots:
Rclone+WebDAV:
16.80s user 1.83s system 15% cpu 1:58.44 total
Rclone+SFTP:
16.49s user 1.40s system 4% cpu 6:35.34 total
Rclone+SMB:
17.27s user 1.80s system 3% cpu 9:26.69 total
Restic native SFTP:
18.03s user 1.74s system 9% cpu 3:39.28 total
@MichaelEischer Looks like 0.17.3 is much faster! Probably still going to stick with Rclone+WebDAV, but that’s good to know
I still wish Restic did WebDAV natively, but I get not reinventing the wheel when Rclone already does it. Just seems to be a much more efficient protocol, though.
Thanks for the comparison. The interesting thing with these numbers is that they stress test how many read operations are possible in a given time. They barely test the data transfer rate for each backends.
Reading a blob via sftp requires three steps: open a filehandle, read the blob, close the file again. For webdav it’s probably only a single request.
I’d expect most of the performance difference to disappear when using a cache as the number of read operations will be much lower.
Yes, the performance seems the most marked when it comes to enumerating files - even just as a MacOS mount (native, not Restic), the WebDAV lists things much, much more quickly. But actual download speeds are about the same. HOWEVER… in general, I get a whopping 4MB down, 800KB up, on my home internet connection… so me testing download speeds for y’all is kind of a moot point 
Now, I have a consistent gigabit connection at work, but we have blocked outgoing SFTP connections. So there’s no real way for me to compare download speeds in a way that would be meaningful to anyone here lol. Both SFTP and WebDAV are going to easily saturate 4MB/s at home, and I can’t test at work. So enumeration operations seemed to be the most reasonable test.
But I’d be interested if someone else decided to give it a try!
Now something I can do is test enumerations with cache… it’s going to be the same command as above, just without the --no-cache switch… sooo, here goes!
First round:
Rclone+WebDAV:
24.99s user 1.94s system 193% cpu 13.908 total
Rclone+SFTP:
26.30s user 2.25s system 139% cpu 20.400 total
Rclone+SMB:
25.39s user 2.20s system 100% cpu 27.438 total
Restic native SFTP:
28.26s user 2.01s system 141% cpu 21.329 total
Second round:
Rclone+WebDAV:
28.12s user 2.17s system 199% cpu 15.210 total
Rclone+SFTP:
26.78s user 2.00s system 155% cpu 18.464 total
Rclone+SMB:
26.89s user 1.91s system 106% cpu 27.028 total
Restic native SFTP:
26.26s user 2.05s system 135% cpu 20.969 total
Third round
Rclone+WebDAV:
27.86s user 2.49s system 205% cpu 14.776 total
Rclone+SFTP:
27.00s user 2.18s system 146% cpu 19.976 total
Rclone+SMB:
24.13s user 1.59s system 102% cpu 25.031 total
Restic native SFTP:
27.18s user 2.34s system 137% cpu 21.498 total
Conclusions
WebDAV FTW! 
(by a much smaller margin, as expected)
Ps. Ignore my system CPU - in the middle of rebuilding a SnapRAID array lol
Oh, and if anyone else wants to try their hand at it, this is how I do it:
export RESTIC_REPOSITORY=rclone:hetz-dav:restic-db && \
time restic diff 0ee9c1ad 47d80a02 -q && \
export RESTIC_REPOSITORY=rclone:hetz-sftp:restic-db && \
time restic diff 0ee9c1ad 47d80a02 -q && \
export RESTIC_REPOSITORY=rclone:hetz-smb:backup/restic-db && \
time restic diff 0ee9c1ad 47d80a02 -q && \
export RESTIC_REPOSITORY=sftp://xxx@xxx.your-storagebox.de/restic-db && \
time restic diff 0ee9c1ad 47d80a02 -q
Just make an rclone connection for each method, and replace the repository values. Then change the restic command and/or snapshot IDs to whatever you’d like. Note on testing SMB with Hetzner, you do need the “backup” share in the URL. Also optionally add --no-cache. If you do use cache, I’d recommend running it at least twice, so that everything is nice and pre-cached for the 2nd (or more) test.
ITs a shame they took away the 2TB option — I recall it had a really good price/value ratio and I was eyeing it for a while… 1TB isn’t enough for my needs but 5TB, despite even better value, is a bit wasteful for me.
Meanwhile I’m wishing they would have a sale on the 10TB box, cause I’m very close to filling my 5TB one. Probably need to prune some stuff… 
I like to add I am using restic iwth hetzner on a 1G line…
The upload is fine the download is like 200M Max… ( I test download the files form a sftp client and the speed are the same).
I am using sftp with 0.17.3 of restic.