Does Restic support SFTP as a backup *source*?

Getting Help
1

I want to backup using a computer running Restic, which is acting like a middle man between the source and destination:

+--------------------------------+
|  SOURCE:                       |
|  NAS Server                    |
+--------------------------------+
               |
               v
+-----------------------------------+
|  AUTHENTICATION & DATA TRANSFER:  |
|  SFTP                             |
+-----------------------------------+
               |
               v
+--------------------------------+
|  Computer running Restic       |
+--------------------------------+
               |
               v
+--------------------------------+
|  DESTINATION:                  |
|  Restic repo, cloud bucket     |
+--------------------------------+

Syntactically, I expect it to work something like this:

restic -r s3:s3.amazonaws.com/bucket_name --verbose backup sftp:user@host:/srv/source

TLDR; Does Restic support this type of use case, where the source is not local to the machine running Restic?

Edit: so many edits due to trying to get the ASCII flowchart designed properly.

2

No, it does not. You’ll have to present the SFTP mount as a FUSE filesystem or similar so that you can feed restic a local path to back up.

To clarify; The source you are backing up does not have to reside on the same machine as restic runs on. But you have to feed restic a filesystem path.

1 Like
3

Allow “pulling” backups from remote servers #299 is a good read for this type of use case.

3 Likes
4

Whoa, mind blown! THANK YOU. I didn’t even know I could use macFUSE to mount an SFTP source! For thread posterity, I found this guide to set up macFUSE with SFTP: How to mount SFTP as a drive on Mac in 5 easy steps

I wonder: is it possible to auto-mount on boot? I assume the auth key would need to be stored somewhere. This would make scheduled backups a bit more robust, not having to worry about mounting after every reboot. Not a big priority, but if you have any ideas I’d love to know!

5

This is an amazing find, thank you for the link.

This comment gives me pause:

sftp/sshfs is way too slow for big backups. Getting a file list over sshfs is multiple orders of magnitude slower. We are talking minutes instead of seconds. So even if no files have been changed, it needs a long time to just detect if anything is changed.

In the context of @rawtaz 's suggestion using macFUSE to mount locally, would these concerns apply? I’ll be backing up > 1 TB that doesn’t change very often; incremental adds, and moving files to different folders.

6

Try it and see :slight_smile: That’s the only way to know how it works for your particular case.

Note that there is a --no-scan option (“do not run scanner to estimate size of backup”) you can give to restic backup which will take away some of the I/O for you. Just make sure you run restic version 0.15.1 or higher.

Can’t you just run restic on the SFTP server instead?

1 Like
7

Fair enough! :slight_smile:

Note that there is a --no-scan option which will take away some of the I/O

Briliant. Thanks for the heads up. I haven’t broken out of “beginner usage”, so I would have never stumbled upon that tuning parameter. Awesome!

Can’t you just run restic on the SFTP server instead?

This is a great question. The “server” to be backed up is a Synology NAS on the local network.

8

In my experience, the Synology ssh is very unreliable. I use a docker container of openssh from linuxserver.io.

https://fleet.linuxserver.io/image?name=linuxserver/openssh-server

1 Like
9

Whoa, I appreciate your perspective. Until this point, I’ve only used AFS and SMB. SMB was terrible, but AFS has been acceptable. Since learning about SSHFS I’ve been wondering if it would be better than AFS for regular mounting of my Synology NAS.

Would you mind expanding on your experience with SSH? Are you mounting with SSHFS?

Eager to check out that docker container. Very, very kind of you to share. Thank you.

EDIT: do I even need to use SSHFS? Can I just use AFP to mount the Synology NAS to my laptop, and point it to the local /Volumes/whatever where AFP has mounted it? Is there any reason at all to use SSHFS?

10

I do not see a reason to use SSHFS when you have a Mac that can use AFP instead. And yes, you can point restic to the path where your AFP-mounted data is to back it up.

11

I’m using it as a destination for SFTP backups from Arq on macOS.
The built-in ssh works fine for logging into the system and running commands, but not for backing up to it using sftp and Arq,