Pack contains errors, ciphertext verification failed

Getting Help
1

I run restic check --read-data on my repository today and I got an error:

Pack 5ef6ac9a contains 1 errors: [blob 6: ciphertext verification failed]

Fatal: repository contains errors

Why does this issue happen and how to fix it?

The version is 0.9.6 (I don’t know why apt-get finds 0.9.6 to be the latest version not 0.12).

I found the file starting with 5ef6ac9a and ran:

restic -r sftp:repo restore 5ef6ac9a --target ~/

and I get:

invalid id, no matching ID found

When I run:

restic -r sftp:repo cat blob 5ef6ac9a

I get:

Fatal: blob not found

Thank you!

3

That error usually indicates a problem on the host creating the backup. (Or in some cases on the host running the checks. Does restic check --read-data-subset 95/256 continue to return that error?).

This error can only occur when the affected blob was damaged during creation of the pack file (or while checking it). This is usually caused by bitflips in memory, so please run a memtest on the affected host. Which kernel version are you using? There have been problems with older version of Linux 5.2-5.4, see also Troubleshooting unreproducible consistency errors - #19 by cyounkins.

Repairing the repository would then either amount to following the steps in Hash does not match id during prune · Issue #1999 · restic/restic · GitHub or to use restic find --pack 5ef6ac9a to find the affected snapshots and remove them.

apt-get uses the latest version packaged for your distribution. That version is frequently older than the latest version available for some software.

restore expects a snapshot ID, and cat blob expects a blob ID. The ID returned by check is a pack ID.

1 Like
4

I forced-updated restic to version 0.12 and this time I encountered no error with the same command restic check --read-data.

  • Are these random errors that occur due to bit flips during the check process, meaning that, if the same command is run again there might be no errors?

  • How much data is damaged if one blub in a pack is corrupted? If it’s small (like it impacts only one file), we can ignore a couple of corrupted blubs.

  • How come newer version of restic encountered no error?

Thanks

5

Sorry for the very late reply.

If the error disappeared then its usually some problem with the host verifying the backup, assuming that the damaged blob was not removed in the meantime by running prune.

That depends very much on the damaged blob. It might as well be a blob only used for a single file in a single snapshot or a blob used by hundreds of file in every snapshot. You could use restic find --pack 5ef6ac9a to get an overapproximation of possibly affected files. This will output too many files as it will report every file which uses any of the blobs in the pack file and not just for the damaged blob.

My guess would be that it was either a bitflip while checking the backup or that the pack file has vanished in the meantime.