Restic backup behaviour when interrupted - s3 backend

Hi,

I’m using restic to backup our databases to an s3 compatible object storage bucket in the cloud. In most cases, the database is one single file, size > 10 GB.

As I’m backing up directly to the cloud over the internet, I’m thinking about what happens when the internet connection is suddenly dropped or interrupted during backup. What would you recommend as the most sure way to check if the backup was successful?

To start the backup, I’m running a “restic backup …” command: will this always return an error in such a case? Or do you recommend to add a “restic check …” command after the backup each time to be sure everything went fine?

Btw, my experiences with restic are really great so far! Using it on Windows Server machines from Powershell scripts.

Thanks in advance!

If a backup is interrupted you can just restart it - restic will only upload the parts of the data that hasn’t already been uploaded.

When restic finishes a backup, it will tell you that it created a snapshot, and what the new snapshot’s ID is. I would scan the output for this information and only conclude that the backup has been finished when I find that information. That should be enough, no need to run a check just to know if a backup finished, that will be expensive in the long run.

@rawtaz Thanks for the info - didn’t know you can just restart it!

ATM I’m checking restic’s exit code: if it’s non-zero I’m considering the backup as failed. Would that be enough? I can add a regex match on the output to find the snapshot id too if it’s needed.

Yes, if it’s not zero you can restart.

Thanks for the reply’s, I did more testing yesterday and my findings about backup resiliency are very good:

  • if your internet connection suddenly drops, restic keeps retrying and recovers automatically
  • if your backup source disappears during backup, restic stops; restarting backup after reconnecting backup source didn’t continue where it left off but it nicely backed up again; (duplicate) data blobs already written were removed with prune

All together a very good and resilient backup tool!

Btw, adding the --json option for the restic backup command would be very handy (i.e. adding the ability to run it non-interactively using a script). The --json option already works for the snapshots and stats command, which is very powerful for scripting.

Since restic 0.9.5 the --json flag is actually supported by the backup command and produces a stream of json messages.

It’s generally recommended to run the check command from time to time to ensure that no data corruption (e.g. due to hardware problems) has sneaked in.