Multiple hosts adding data to same repo

Getting Help
#1

Hi,

I wonder what happens when multiple hosts are running backup simultaneously to the same repo.

For example:

  • file1 (size 10 GB)
  • file2 (size 500 GB)
  • file3 (size 5 GB)

Host1 is running backup of all 3 files and just starts sending file2 (and will finish in more than a day)
Host2 is running backup of only file3 (and will complete it much sooner than Host1 will finish file2)

Will Host1 skip file3 since Host2 will upload it in the meantime? Or will Host1 not be aware that Host2 uploaded the file3 already and will create a duplicate, that later can be removed with prune?

I am using restic 0.9.6 compiled with go1.13.4 on linux/amd64.

Thanks

#2

The latter variants is what happens. Each backup run reads the repository index when starting the backup and the continues independently of other backup runs. If Host1 didn’t know some or all of the data blocks of file3 at the start of the backup run, then it will later on upload those blocks no matter what other backups uploaded to the repository in the meantime. These duplicate blocks will then be removed by the next prune run.

#3

Thank you for clarifying this and I still think Restic is a great tool.

Cheers