Potential Issues with Concurrent Execution of Restic Commands

Getting Help
1

Hi everyone,

I have a question about running multiple Restic commands at the same time. Are there any known issues or risks if the restic backup, restic forget, and restic check commands are executed simultaneously on the same repository? Any insights or experiences with this would be greatly appreciated!

Thanks!

2

hello @hmsayem,
restic allows certain operations in parallel and controls it via a lock mechanism.

You can read about it here:
https://restic.readthedocs.io/en/stable/100_references.html#locks

Unfortunately I cannot find the paragraph where it was listed which operations require an exclusive lock. Exclusive lock will restrict repository operations, i.e. no other operation can be allowed during that time.
IIRC then restic check and prune require an exclusive lock. Backups can be done in parallel.

If you do not have control over the timing in which the commands are started, then I would recommend to include checks for the repository lock status.

3

If I understand correctly:

  1. If a command is running with an exclusive lock, any other command requiring the same lock will fail.
  2. While a command is running, any other command that needs an exclusive lock will also fail.

My main concern isn’t about commands failing, but could this potentially cause issues with the repository?

Also, if the restic check command runs with the --no-lock option and, during this, a restic prune command is executed (or vice versa), could this lead to problems? Would you recommend avoid using the --no-lock option in these situations?

4

check does not modify the repository. So the worst that can happen is that prune removes some files and check later on complains about a broken repository. Running check again in this case will then report the correct result.

The rules are actually slightly more strict. If a repository is exclusively locked, then no other command can lock the repository. That is, for example backup and prune cannot run concurrently at the moment.

5

check does not modify the repository. So the worst that can happen is that prune removes some files and check later on complains about a broken repository. Running check again in this case will then report the correct result.

So It is fine to run multiple commands using --no-lock (eg. restic stats, restic snapshots etc.) alongside a command that requires a lock (either exclusive or non-exclusive). Correct?

6

May I ask why you insist on using --no-lock flag? Restic today is designed to run with locking, exclusive and non-exclusive. By default it takes care of applying correct locks and respecting them. Unless you have deep understanding of restic under the hood operations and real need it is not necessary to control locking manually.

7

It won’t damage the repository, but can return wrong data.

8

It was introduced to to prevent situations where a command fails to release a lock on a repository, Appreciate your insight on this!