Snapshot integrity

Getting Help
1

Hi,
The documentation talks about the integrity of a repository. Is it possible only to check the integrity of a snapshot?

Thanks,
RG

2

restic check also checks this.
To be more precise, it checks that the trees structure referenced in all snapshots exist and can be loaded and that all file contents are correctly referenced in the index. If you want to check that also the file contents can still be loaded, you have to specify one of the -read-data options.

3

Hi Alex,
Thanks for the reply. But my use-case is different. My repo takes above 1 hour to be checked, but my incremental backups are small. I would like to be able to check only the latest snapshot (I suppose in a few minutes tops) as opposed to full repository.
Is that possible?

RG

4

There is no possibility ATM to only check a part of the repo.

What exactly do do you want to check after each backup run? Which failure scenario do you think of that restic should report?

Do you use the latest version of restic? There have been quite some speed improvements, also with respect to the check command.

Moreover, if you do a follow-up backup (note, there are no true “incremental” backups in restic, but all snapshots reference a full backup), the parent already gets kind of checked automatically, at least if the tree structure can still be loaded.

2 Likes
5

The only way I can think to check an individual snapshot would be to actually restore it somewhere.

6

Thanks for the replies. I guess what I was trying to achieve is different than a snapshot check. Now I got it that if I do a snapshot check, I do a full check for all files that belong to that snapshot. I was thinking more around the lines of checking what was recently uploaded, a sort of a “diff check” between the latest snapshot and the one before it. My idea was simply to validate that whatever was uploaded in the last backup is integral.
Right now, I am backing up to Box.com, using the app provided by them. When I upload a file to /Users/renan/Box/repo, the file first get cached locally in my machine, and then is pushed to box.com. So, when restic backup reports success after backing up, the success was to write my file locally, not remotelly. This eventually happens later on and its managed by Box app. My first backup is around 1TB, and then each day it grows around 1 GB. I would like once a day to check only what was uploaded during that particular day, not the full repository associated with a particular snapshot.
As a workaround, if I do check --read-data-subset=100/100, does it mean I will check the most recent 1% files that were uploaded?

7

No, it would test files in the repo which start with chosen letters. Rest has no of finding out what was recently uploaded as it does only use the filename and filesize but not the timestamps of the files in the repo.

If you know which files have been recently uploaded (e.g. by a script that evaluate the timstamps of the files), it should be IMO sufficient to check the SHA256 sum of those successfully generated files. This should be equal to the filename and hence can be checked without calling restic.

8

If you know which files have been recently uploaded (e.g. by a script that evaluate the timstamps of the files), it should be IMO sufficient to check the SHA256 sum of those successfully generated files. This should be equal to the filename and hence can be checked without calling restic.

Thanks Alex,

I will probably make a script like you said. Question: If during this check, I realise the SHA256 sum is different than file name, what would be the course of action to get my repo integral again?

9

Always delete this broken file. The next action depends a bit on the type of file:

  • index file: run restic rebuild-index
  • pack file: Run restic rebuild-index and redo the last backup. If there is no change (in the files/dirs that were stored in the broken repo file), this will be very fast and heal the repository.
  • snapshot file: Just redo the backup

Always run restic check after repairing a repo!

If you have a broken file and that contents are no longer present, this is getting more complicated… :stuck_out_tongue_winking_eye:
restic is so far missing a good troubleshooting FAQ; this is being worked on…

10

Thanks for the support. After doing a full backup on Box.com, I did a restic check and it reported several missing packs. As indicated, I then did restic rebuild-index and restic backup. This time the backup was very quick. To finalise, I did a new restic check, and it reported the repo is integral.

Cheers!

11

Here’s a script I occasionally use to compare SHA256SUMs in the repo. If you wanted to limit this by age, it require only one more term on the find command: -ctime -1 might be what you’re looking for.

#!/bin/bash
# rqc.sh - restic quickcheck
# this script does a sanity check on a restic database to look for data corruption
# hosted locally
BASE_REPO="/mnt/repos/"

#check usage
[ -z "$1" ] && {
echo no repository specified.
echo usage: $0 repository-name 
echo example: $0 home 
exit
}

TEMPFILE=/tmp/rqc-$$

#build a list of files to check.  Only include files with hexadecimal filenames.  Put the list in the sha256sum file format.
for i in `find $BASE_REPO/$1 -regex .*/[0123456789abcdef]\* -type f`; do echo `basename $i` $i >> $TEMPFILE; done

#check all the files and report results.
sha256sum -c --status $TEMPFILE && {
  echo quickcheck of $BASE_REPO/$1 successful
  rm $TEMPFILE
  exit 0
} || {
  echo quickcheck of $BASE_REPO/$1 FAILED!
  rm $TEMPFILE
  exit 1
}
12

Thanks very much David!