Restic + RClone remotes and syncing

Getting Help
1

Greetings, total noob here, apologies for simple question.

I setup rclone and have 3 remotes, now my idea is to:

  • Source data: unraid and 2 of the rclone remotes
  • Destination: 3rd remote

I was reading the rclone restic setup instruction, but it seems like it sets up the remotes as repos, which is OK for the 3rd one that will be repo, but the others will be datasources. Should I be mounting them via rclone instead of serving them to restic?

Follow-up question: I am also thinking of backing up 2 of the remotes first to my unraid, only then to the 3rd remote. What would be the best way to achieve this? I’m thinking running the backup twice (first from 2 remotes to unraid, then from 2 remotes to 3rd remote) would waste ton of bandwidth/time unnecessarily. Perhaps copy the snapshot from unraid to 3rd remote? Is it easy to automate this?

Thanks for any help!

2

As I’ve never heard of the option of restic using a Rclone remote as a SOURCE, I’m doing that exact same thing.

Starting from behind: Easy to automate is depending on your experience. When you have years of experience in writing scripts, that may take less than an hour. But if this is the first script you write, I hope you have some good endurance.
Backing up from the two remotes to your unraid and then backing up to your backup is only advisable if and only if your first backup’s are in cleartext (e.g. using rclone sync, rsync or similar) but not using restic, since restic encrypts the data and every time something changes you generate a totally new blob in restic. Which then has to be backed up, this can produce some overhead, as deduplication suddenly is on a bloblevel and not a filelevel.
If I were you, you either mount that stuff using rclone and run restic (with ignoring inode flags, as they are not preserved) or sync that to your unraid. Unless you have sensitive data, that should not be stored unencrypted on your unraid, then I think your proposed solution is best fitted.

1 Like
3

Thanks a lot for your response, much appreciated!

I ended up doing the following:

  • Sync 2 rclone remote SOURCES to Unraid
  • Backup these 2 sources with restic to my Unraid, that’s my Repo1
  • Backup these 2 source and another Unraid directory to 3rd remote DESTINATION, which is my Repo2.

Probably not as elegant as doing it directly as I’m basically wasting some space for the 2 SOURCEs to be synced to my Unraid before they get backed up to Unraid and RClone REMOTE, but works nicely.

I guess next step would be to mount them via rclone and remove this intermediary step…

Thanks!

1 Like
4

Ok, tried with rclone mount and got nowhere, most likely because of caching issues. I suppose restic wasn’t built for the use-case of backing up from Google Photos to OneDrive, as it doesn’t make much sense to begin with anyway…

So I’m back to using rclone sync to cache the remote first locally, then push them via restic to remotes.

Thanks!

5

So, as I had a similar problem a long time ago and I looked up what my solution was:
I mounted the cloud, with --vfs-cache-mode full and run restic backup --ignore-inode. But the first backup was slow.

6

Thanks, I’ll try that. I used vffs-cache-mode=full before, but not the ignore-inode

The problem is I’m backing up around 120 GB from one remote to another, some files are quite big and I think they overwhelmed the VM cache.