Damaged repository with ciphertext verification failed

Hello,

FYI @damoclark @MichaelEischer just for the sake of being thorough, I tried a restic check --read-data on the alpha (now ext4) external drive with no external power, without the externally powered USB hub, trying to reproduce the disconnects that happened every single time on the formerly NTFS drive. I could not reproduce the issue, I think this means the whole problem came just from the NTFS implementation on Linux.

So, to summarize this whole thread:

Problem

The NTFS on the drive the repository was stored in, was pretty damaged and had a few folders that could not be read, leaving the repository in a broken state.

This damage to the filesystem was likely originated by multiple disconnects/resets of the drive while operating, these could be observed during restic check --read-data command in the dmesg output, but likely occurred during backups as well.

It seemed like this was a hardware issue, it was consistently reproducible but it stopped happening when using an externally powered USB hub in between the drive and the computer, which further hinted at a hardware issue, however, after changing the partition format to ext4, the error cannot be reproduced under the original circumstances, maybe hinting at a problem with NTFS itself instead of the hardware.

Solution

After thoroughly checking that the hardware was not damaged (smartctl tests, badblocks, dd, ddrescue, etc.), the solution was to change the filesystem on the drive from NTFS to ext4 which got rid of all error messages in dmesg.

Once the drive was stable enough to work on, the steps on the troubleshooting guide worked like a charm.

Conclusion

The NTFS implementation on Linux is brittle at best, the utilities to check/repair the FS are not good enough and you end up relying on a windows machine to fix the issue. I wouldn’t recommend using NTFS again for this.

Thanks for all the help and for sharing your expertise with me to help me solve this problem. I’m very happy with the interactions we had and thankful to the community.

1 Like