Backup failed with very old lock

[john.isles]

Hi there,

I have been using restic_0.9.6_linux_amd64 to backup my system to Backblaze every night for many months now without issue. However last night I got the following error:

Fatal: unable to create lock in backend: repository is already locked by PID 6694 on admin by admin_user (UID 1000, GID 1000)
lock was created at 2020-04-20 16:59:47 (892h6m0.943953987s ago)
storage ID 9c2d0aa5

It seems odd that the lock was created over a month ago. Can anyone explain why this happened?
My backups usually take less than a minute (not much changes on my system from day-to-day), so the backup from the night before was definitely not running and there is no process ID 6694.

Thanks,

John Isles

[cdhowie]

Backup doesn’t take an exclusive lock anyway, so two backups can run concurrently. If backup is complaining about a lock then it’s an exclusive lock, which is only taken by some commands.

List of lock rules in documentation

I agree. Here is a table detailing which commands take which type of lock. Shared locks allow any number of other shared locks to exist at the same time, and these commands typically only read or add data. Only one exclusive lock can exist at any time, and it must be the only lock. These commands typically delete information from the repository, or otherwise require it to not change at all while they work. tl;dr: Shared allows other shared stuff to run. Exclusive doesn’t allow anything else …

I’m not sure why a lock would have suddenly appeared. Did you do any other maintenance on your repository within the last day, perhaps from a computer whose clock might not have been set correctly? Are you sure that the backups from previous days actually did run successfully?

[john.isles]

Hi there,

Hi cdhowie, thanks for your reply.
I think I have found my problem. I run restic from a shell script which occasionally runs a “restic check”.
It is this command that is actually generating the error message.
There was an exclusive lock in the repo which did not affect the nightly backup, but did affect the check which runs very occasionally. I have removed the lock and the check now runs without error.

Thanks for you help. The reference to the table of locks and your explanation gave me the “ahha” moment.

John

[cdhowie]

@john.isles

There was an exclusive lock in the repo which did not affect the nightly backup, but did affect the check which runs very occasionally. I have removed the lock and the check now runs without error.

Point of clarification: this would be a shared lock. That would block the check but would not block backups.

Glad you found the issue!

[john.isles]

Thanks for your help cdhowie.
And thanks for a great backup program.
Just for your interest on users of restic, I am currently backing up about 45TB of data.

[cdhowie]

@john.isles

And thanks for a great backup program.

Most of the credit goes to @fd0. Restic is his brainchild. I’ve submitted some patches here and there but that’s it.

@john.isles

Just for your interest on users of restic, I am currently backing up about 45TB of data.

That’s really awesome to hear! We’re always happy to hear that restic is doing its job protecting data.