I’d like to deploy restic as d2d backup solution in a local network with several Linux servers. There should be one backup server holding the restic repository, all other servers should backup their data automatically to the repository on the backup server. Some data needs to be backed up more frequently than others (say, hourly vs. daily), this should be managed automatically by cron jobs.
I plan to use rclone for implementing an append-only strategy (for security reasons, backup clients must not delete or modify existing backups)
I have a test setup now running for some time.
On the backup server I have a repository located under
/data/backup/restic/repo. The backup server is running CentOS 8 with restic 0.9.6. The backup clients run various different Linux distributions with restic 0.9.6.
For all backup clients I have their SSH public key stored in the
$HOME/.ssh/authorized_keys file of the restic user on the server like this:
restrict,command="rclone serve restic --stdio --append-only /data/backup/restic/repo" ssh-rsa AAA...
On the backup clients I set the following environment variables:
export RESTIC_SERVER_ADDRESS="firstname.lastname@example.org" export RESTIC_REPOSITORY=rclone: export RESTIC_PASSWORD_FILE=/etc/restic/repository.pwd
and then I use restic in backup scripts like this:
restic -o rclone.program='ssh $RESTIC_SERVER_ADDRESS' backup /dir
This command then creates a new snapshot in the repository with the host set to the backup client host where the script is running on and the path set to the given directory.
Though this setup works in principle, the backup always seems to fail if backup jobs on different clients are started concurrently at the same time.
I.e. if I have a long running backup job on one client and then start another backup job on another client (both backing up to the same repository on the same server), the jobs just seem to hang as soon as the second job is started.
So far I can only have reliable backups if I make sure to start all the backup jobs on all the backup clients in a strictly sequential, non-overlapping order. Needles to say that this is very hard to achieve in a real-world scenario…
So the question is: does restic support several concurrent backup jobs to the same repository when using SSH/rclone at all?
Thanks for any insight!