SFTP performance

Getting Help
1

Hi,

I started using restic over SFTP over local network, my home server to my desktop PC which has external disk attached for backups.

I don’t have exact numbers, but restic over SFTP for me felt significantly slower than “rsync -e ssh”. I can probably do another several runs and do proper benchmarks if anyone is interested.

My questions are:

  • Did anyone else notice this? I mean would you consider restic over SFTP slow?
  • What can be done to improve the performance? Is this a limitation of SFTP protocol and/or restic repository format?
  • What other protocols could be used that have better performance? I’d prefer something with encryption while on the wire. And I’ll be using this over network with my hosting provider, so something that works fine with high latency would be nice. EDIT. I mean I can start some S3 storage server or HTTP server on my backup machine if that helps and it’s faster than SFTP.

Thanks,
–Coder

2

I would recommend trying rest-server or Minio, both are supported backends directly in restic.

3

In my experience sftp should perform quite well for backups. Or to be a bit more precise: uploads to the server should perform quite well. The problem is currently on the download side, where the throughput of the current sftp backend implementation suffers even when connected of a LAN (I have some changes lying around which would fix the download speed, I just need to find time to clean those up far enough to be suitable for a PR). At the start of an operation restic also has to list files existing on the server which due to the sftp client used by restic is quite latency sensitive and is already slower in a LAN than e.g. when using the rest server.

Just to state this explicitly: the operations like check, prune, restore are currently a bit slow when used over sftp. What do you actually consider as slow? That term can mean pretty much anything.