Partially recreate repository with new backups

Getting Help
1

Hello all,

I have a restic repository in a failed hard drive, completely gone.
This repository was cloned with rclone to a aws s3 bucket with auto tiering. Now that I have to restore the data from aws to a new local drive I wonder if there is another (cheaper) way.

Since I’ve lost only my backups, I can backup everything again on a new repo, made restoring from aws config and keys. This means I’ll get a new repo with almost all data in place, but in different data files.
The idea, after a full backup to this new repo, is to copy all the snapshots and indexes files from aws and perform a check. Ideally all the new data should be already in the repo, while old deleted file should be referenced in indexes, but not available in the data files. Now I can copy just the files with missing data listed by restic check and try again.

Of course copying the data from aws is simpler, but it will cost money and it is not fun…

Can this work? I’m willing to try, but I’m asking in case there is no chance for this to work.

Thanks,
Radel

2

The check command would complain about nearly ALL pack files that exist in the S3 repository. check does not only verify that one copy of each blob exists but rather that every pack file referenced in the index does indeed exist.

While the approach in general could work, you’d at least have to modify the check command to also provide you with the necessary information about which pack files are actually necessary. Depending on how much your backup has changed over time and how often you’ve ran prune, there’s a certain chance that you still have to download a large part of the repository.

3

Lately has been important to me to try and fix this Frankenstein repository (due to errors in its backup), so I would try another possible approach.

After a full new backup I copied snapshots file from the copy repository to the new one. Of course restic check complains about missing data.

Now I’m starting, snapshots per snapshots, a tree traversal, and I make a list of needed “content” ids (for files) and I descend into each subtree recursively.

I then will make a diff between restic list blobs and this list of needed blobs, copying the missing one from the data repository. It will be painful, but I want to know if I can end up with a clean repository.

I’m also skipping duplicated subtrees in order to save some compute time.

Now I’m not just trying to save some bucks, but this seems one of the last chances to recover my repo…

Am I on the right track?

Note that all of this is made with bash scripts, jq for looking into json structures and a lot of restic list and restic cat, because I cannot write in go… :frowning:

4

Done!
In case someone is interested I’ll quickly report what was done.

I created a new repo with the same config file of the original repo.
After that, full backups. After that I copied the old snapshots to the new repo and run restic check. It complained about blob xxx not found in index.

Consider I got tons of this errors: I collected all the missing blobs in a file. For each blob I looked for the corresponding pack in the indexes, collecting a list of missing packs.

I soon realized that accessing the indexes from restic is too slow, so I extracted all the indexes in a file and used jq to look for the packs number.

jq --slurpfile BLOBS blobs.json \
   '.packs[] | select(.blobs[].id as $id | $BLOBS[0] | index($id)) | .id' all_indexes.json \
   > needed_packs.txt

The missing packs, about 70000 files, were restored from glacier and added to the data directory.

restic repair index and restic check…

Now I have a working repository, made mostly (60%) from new backups and with just the missing data (40%) downloaded from AWS.

Given how much AWS bills for bandwidth I think it was worth it!

2 Likes
5

Awesome work! :slight_smile: