Hi there, what method would you recommend for backing up a remote folder (e.g. one hosted with a cloud service provider or on an SFTP drive) to a local backup on my laptop? I imagine Restic is used far more option for remote backups of local folders, but in my case, I’d like to be able to store a backup of my SFTP drive on an external drive connected to my computer.
One option is to mount the SFTP drive via Rclone, then use Restic to back up the mounted folder to my external drive. This works fine, but if there’s a way to bypass the mounting operation, that would be ideal (since mounting can take up a decent amount of hard drive space, at least in the near term, if I use certain vfs-cache-mode settings.)
Another option, in my case, would be to just connect the external drive to the SFTP server. That will generally work fine, but it would be nice to be able to update the external drive’s contents when I’m not able to access the server.
I thought that I could try reversing the syntax of the code described in this section of the Rclone documentation to achieve this setup, as shown in the following example:
We used to have a similar situation with an FTP (!) server, and this is what we did. It wasn’t crazy fast or anything, but it ran at 2AM so nobody cared.
Today, Restic can only back up from the local filesystem, so to back up something remote you need to make it appear local, which is exactly what rclone mount does.
You could also experiment with sshfs and see how performance and disk usage compares.
I use a reverse tunnel in this case. Here’s a command for backing up to the local machine that is running rest-server but it works with sftp as well, of course:
It basically says that local port 8000 (rest-server) is opened on the remote machine as 1337. Restic then backs up to that port on the remote machine and is tunneled to port 8000 on the local machine.
Note that the restic password is also just set on the local machine. In this case the host-to-be-backed-up is exposed to the internet and the local machine is “safe” behind a firewall. If the exposed machine is compromised, there is no connection to the backup except during the backup process.
Beware that using --password-command this way will leak the repo password to all processes on both machines, and may show up in shell history files. It also may fail if the password contains special characters interpreted by the target machine’s shell.
When I moved from zbackup to restic, I also missed that functionality.
With zbackup, I basically did the following ssh $REMOTEHOST tar cf - / | zbackup $DIRECTORY. (Note that the tar command ran remotely on the machine which had to be backed up while zbackup ran on my backup server. So no need to install any application on the machines I wanted to backup)
However, I don’t have a good solution to achieve this with restic. The option to mount the remote machine somehow has already been mentioned. Of course, you can also do ssh $REMOTEHOST tar cf - / | restic backup --stdin, but it has 2 drawbacks: You will 1. always send the whole data over the network and 2. you only have a single large tar file in the snapshot.
A nice feature would be, if restic backup could parse a tar stream. You would still send the whole data over the network every time - but at least you would have individual files in each snapshot. But I guess there are higher priority features
