Multiple rclone instances

Getting Help
1

I run restic via a script and multiple schedulers. The backups utilize OpenDrive via rclone - all the same repository but different folders & tags.

It frequently occurs that multiple instances will be running (for different folders - only one instance for a given folder is running). This means multiple copies of rclone will be running as well. Is it possible/desirable to instead run a single copy of rclone that would service multiple restic instances?

2

I did something like git-like public hosting for my backups. Each machine uses its own ssh key (this can be easily automated). On the backup server I run rclone as a restic backend. Each server connects as restic user.

The backups are located in /srv/backup/HOST, to force the use of rclone I put this on the authorized_keys file:

no-X11-forwarding,no-port-forwarding,no-agent-forwarding,no-pty,no-user-rc,command="/usr/bin/rclone serve restic --stdio /srv/backup/HOST" ssh-ed25519 ... SERVER1

If you want a better security you can make sure restic user cannot modify its own key file by adding this in sshd_config

Match User restic
   AuthenticationMethods publickey
   PasswordAuthentication no
   X11Forwarding no
   AllowTcpForwarding no
   AllowAgentForwarding no
   AllowStreamLocalForwarding no
   PermitTTY no
   PermitTunnel no
   PermitUserRC no
   AuthorizedKeysFile /etc/ssh/authorized_keys/%u
   MaxSessions 10

And put all the ssh key in /etc/ssh/authorized_keys/restic owned by root.restic with perms 0640.

Now you have to invoke restic using:

restic --repo "rclone:" --option "rclone.program=ssh -i /path/to/ssh/key restic@BACKUP_SERVER" ...

Hope that helps.

3

Thanks - but that doesn’t answer my question.

I’m using OpenDrive - which means no ability to run remote servers. The only means of communication are their API (which rclone supports) or WebDav. So…I’m communicating via rclone.

My question is if I’m running multiple simulataneous backups, to the same repository, presently that means multiple rclone instances. This works - I’m asking if it would be better to create a single rclone instance that the multiple restic instances would use.

4

My suspicion without testing is that it would be better as any caching rclone performs would be more effective; all restic instances could share that cache.

1 Like
5

So now the question is - if rclone is typically run by restic - how do I start an instance of rclone that will service multiple restic instances, and how do I tell each restic instance to use that instead of starting its own rclone?

1 Like
6

You could run rclone serve restic as a systemd service, and then direct your restic clients to use the rest: local endpoint.

1 Like