Restic check: dealing with `error for tree` / `blob has null id`

When running restic check with the latest beta build on a repository several errors are shown:

TMPDIR=/path/to/cache restic --cacert /path/to/cert.pem --verbose check --read-data-subset 7%
using temporary cache in /path/to/cache/restic-check-cache-1508918575
repository 05ee4855 opened (version 2, compression level auto)
created new cache in /path/to/cache/restic-check-cache-1508918575
create exclusive lock for repository
load indexes
check all packs
check snapshots, trees and blobs
error for tree 3dfe2312:apshots
  tree 3dfe2312d75f7041de8c1de7459ab4441d70ce58a14edfe73739562fa493c0b4: file "<redacted>" blob 1279 has null ID
error for tree 8cd160fc:apshots
  tree 8cd160fc99faeb0863049073b5fb4f06adc24abcda866cbce13e4f97c5ed3f78: file "<redacted>" blob 61 has null ID
[10:37] 100.00%  16 / 16 snapshots
read 7.0% of data packs
[1:13:23] 100.00%  58934 / 58934 packs
Fatal: repository contains errors

How should one deal with such errors?
It’s also odd, that 3dfe2312:apshots truncates the word “snapshots”.

I assume that error is also reported by restic 0.14.0?

Then use restic find --tree 3dfe2312d75f7041de8c1de7459ab4441d70ce58a14edfe73739562fa493c0b4 8cd160fc99faeb0863049073b5fb4f06adc24abcda866cbce13e4f97c5ed3f78 to find the affected snapshots. When were these created? Do you know which restic version was used to create them?

That’s a known bug of the progress printing code. check ouput appears corrupted · Issue #3381 · restic/restic · GitHub

1 Like

It seems like this is the same issue like https://github.com/restic/restic/issues/3999 but for the the sake of completeness, this is the output:

restic --cacert /path/to/file.pem find --tree 3dfe2312d75f7041de8c1de7459ab4441d70ce58a14edfe73739562fa493c0b4 8cd160fc99faeb0863049073b5fb4f06adc24abcda866cbce13e4f97c5ed3f78
repository 05ee4855 opened (version 2, compression level auto)
created new cache in /root/.cache/restic
Found tree 3dfe2312d75f7041de8c1de7459ab4441d70ce58a14edfe73739562fa493c0b4
 ... path <redacted>
 ... in snapshot f0f8342f (2022-11-01 19:43:04)
Found tree <redacted>
 ... path /var/lib/mysql_binlog
 ... in snapshot f0f8342f (2022-11-01 19:43:04)

There are also the error: parts of <file> not found in the repository index; storing the file again messages when running a backup with the exact same files restic check complains about.

Thanks for the hint about https://github.com/restic/restic/issues/3381 :+1:

This has apparently been solved with archiver: Fix race condition resulting in files containing null IDs by MichaelEischer · Pull Request #4019 · restic/restic · GitHub

Would it make sense to introduce a “Troubleshooting” section in the docs which covers some possible errors one might encounter and how to deal with them appropriately?

Hmm, that specific error only affected prerelease versions of restic, thus it would mostly just clutter the throubleshooting section. For the short term I’d prefer to stick with pointing users towards Recover from broken pack file · Issue #828 · restic/restic · GitHub .

That is until I get around to push Add repair command by aawsome · Pull Request #2876 · restic/restic · GitHub forward. Then the steps to repair a repository will most of the time be rebuild-index, backup [...], rewrite --repair and finally check. No more manually trying to figure out which snapshots are damaged. (That would replace both route 2 and route 3 in issue 828)

What I’m a bit worried about is that this could reduce feedback regarding repository corruptions. We can only fix / harden restic for things we know about.

1 Like