B2 backup "Load(<lock/xxx> ... b2_download_file_by_name: 404"

Getting Help
1

Issue
I have a scheduled backup that runs daily. It targets B2 for the backend. I’ve found that it frequently (once or twice a week) fails with errors indicating that it cannot find the lock file. The backups always succeed, but it produces errors that get flagged by my monitoring. From what I’ve seen the retry logic continues until success, and typically takes about half a min to resolve the issue. From what I can tell, it doesn’t appear to be related to internet connectivity… but it certainly could be.

Questions
Is there a way to suppress these errors so they don’t pollute my log files? I don’t really care if there’s a temporary error, as long as it recovers and the backup is successful.

Alternatively, is there something I can do avoid these errors in the first place?

Many thanks!

Version

restic 0.16.4 compiled with go1.21.6 on linux/amd64

Command

restic backup --limit-upload=300 --exclude-caches --exclude-if-present .nobackup --exclude-if-present .resticignore --exclude-file /tmp/tmp.leUfmAfVKu --tag mytag /location1 /location2 /location3

Environment vars

export B2_ACCOUNT_ID=‘numbers’
export B2_ACCOUNT_KEY=‘key’
export RESTIC_REPOSITORY=‘b2:restic’
export RESTIC_PASSWORD=‘password’`

Output

Load(<lock/cdfd940510>, 0, 0) returned error, retrying after 538.711416ms: b2_download_file_by_name: 404: File with such name does not exist.
Load(<lock/cdfd940510>, 0, 0) returned error, retrying after 951.476121ms: b2_download_file_by_name: 404: File with such name does not exist.
Load(<lock/cdfd940510>, 0, 0) returned error, retrying after 901.793845ms: b2_download_file_by_name: 404: File with such name does not exist.
Load(<lock/cdfd940510>, 0, 0) returned error, retrying after 2.360498931s: b2_download_file_by_name: 404: File with such name does not exist.
Load(<lock/cdfd940510>, 0, 0) returned error, retrying after 3.540423034s: b2_download_file_by_name: 404: File with such name does not exist.
Load(<lock/cdfd940510>, 0, 0) returned error, retrying after 5.600433689s: b2_download_file_by_name: 404: File with such name does not exist.
Load(<lock/cdfd940510>, 0, 0) returned error, retrying after 3.147820101s: b2_download_file_by_name: 404: File with such name does not exist.
Load(<lock/cdfd940510>, 0, 0) returned error, retrying after 4.528310464s: b2_download_file_by_name: 404: File with such name does not exist.
Load(<lock/cdfd940510>, 0, 0) returned error, retrying after 8.192409453s: b2_download_file_by_name: 404: File with such name does not exist.
Load(<lock/cdfd940510>, 0, 0) returned error, retrying after 23.584056688s: b2_download_file_by_name: 404: File with such name does not exist.
using parent snapshot d76e22bd

Files: 0 new, 0 changed, 205353 unmodified
Dirs: 0 new, 21086 changed, 0 unmodified
Added to the repository: 118.703 MiB (35.427 MiB stored)

processed 205353 files, 827.956 GiB in 3:57
snapshot 337b4d61 saved

2

The recommendation is to use the S3 backend, try that and see if it fixes the issue:

image
image728×385 45.1 KB

3

Oh, that’s interesting! Thanks for the tip!

When did that recommendation change? I must have missed that.

I’ll give that a shot and see if it changes anything. Any thing to look out for when changing between the B2 and S2 access methods?

4

Not sure - I’ve been using B2 for the last ~2 years and this has always been the recommendation I think.

(I still use the B2 backend… yes I’m asking for trouble)

1 Like
5

Unlikely. The underlying issue seems to be that it can take a moment between deleting a file and B2 no longer including it in the file listing.

Restic 0.17 will reduce the amount of such log messages (although it requires a bugfix: Intermittent - unable to create lock in backend: circuit breaker open for file <lock/XXXXXXXXXX> · Issue #5005 · restic/restic · GitHub ). Currently it’s not possible to hide those warnings, they are unfortunately too important for debugging to just remove them. We’ll have to rework the log output at some point (no idea when that will be) which will probably add some way to hide the warnings and just log them somewhere.

2 Likes