How to create par2 parity files for a Restic repository

I would like to generate parchive par2 parity files for long-term storage of a restic repository. This helps to recover from errors in case repository is corrupted. Restic currently doesn’t have error correction feature (Duplicity is the only backup app that generates par2 files AFAIK).

Does anyone know what’s the best way to create these par2 files? Should I write a bash script to recursively create par2 files for each file in the restic depository? Or should I generate a tar file from the whole restic repository and then create Par2 files for that one file? But this increases the required space by factor 2 since tarball must be saved too (a tarball generated 10 years later from the same repository may not be same as today’s tarball due to software updates).

Anyone has dealt with error correction with Restic?

3 Likes

That’s the way I do it. It means that it’s nice and quick to make par2 files for the newest additions after each backup (just find any recently-created files). I don’t tend to prune much, so am happy for the odd par2 file to hang around even when the corresponding restic file has been deleted. Otherwise, just make a script to delete the unused par2 files after a prune operation.

In order to avoid making a mess in my repo directory (although I don’t think restic would care if I did), I have a corresponding directory tree just for the par2 files.

My scripts are rather amateurish, and proper engineers would probably balk at them. Happy to share if needed, but it would be better to have more elegant examples in the forum if anyone has any? :slight_smile:

2 Likes

How do you convince par2create to write the par2 files to a different location? My par2 (debian10, 0.8.0) ignores my different name/location :frowning:

Yes, I seem to recall that was a bit of a hassle. I think you have to be careful from where you execute the command. My script does the following (I don’t know which bits are critical, and which are just how I wrote it)…
Assuming you want to make a parchive of the file:
/backup/restic_repo/data/c4/c4048768c4bd77232a7ffc5c4c3671b13e0234c246c80ceabbcd1a68365d1c58
into the file:
/backup/restic_par/restic_repo/data/c4/c4048768c4bd77232a7ffc5c4c3671b13e0234c246c80ceabbcd1a68365d1c58-5.par2

Then run the following commands:

cd /backup
par2create -B ./ -p -n1 -r5 /backup/restic-par/restic-repo/data/c4/c4048768c4bd77232a7ffc5c4c3671b13e0234c246c80ceabbcd1a68365d1c58 restic-repo/data/c4/c4048768c4bd77232a7ffc5c4c3671b13e0234c246c80ceabbcd1a68365d1c58
My full script is below. Use at your own risk :)
#!/bin/bash

#paths to the repo and the parchive directory
rootpath=/path/below/repo
repodir=restic_repo
pardir=restic_repo_parchive

#percentage redundancy desired
PERC=5

#oldest file for which to create parchive files (-3 = 3 days before present)
mtime=-3

cd $rootpath
#find any files changed within $mtime days and pipe them into the do loop 
find $repodir -type f -mtime $mtime -print0 | while IFS= read -r -d '' FILE
do
        DIR=$(dirname "${FILE}")
        DIR_OUT="$rootpath/$pardir/${DIR}"
        FILENAME=$(basename "${FILE}")
        PAR_ROOT_OUT="$DIR_OUT/${FILENAME}"
        FILE_OUT="$DIR_OUT/${FILENAME}"-$PERC.par2
#create directory if it doesn't exist
        if [[ ! -d $DIR_OUT ]]; then
                echo making directory "$DIR_OUT"
                mkdir -p "$DIR_OUT"
        fi 
#create par file if it doesn't exist
        if [ ! -f "$FILE_OUT" ]; then
                echo executing: par2create -B ./ -p -n1 -r$PERC "${PAR_ROOT_OUT}" "${FILE}"
                echo in `pwd`
                par2create -B ./ -q -n1 -r$PERC "${PAR_ROOT_OUT}" "${FILE}"
#remove the unnecessary file created by par2create
                rm "${PAR_ROOT_OUT}".par2
#and rename the other file to the correct name
                mv "${PAR_ROOT_OUT}".vol*par2 "$FILE_OUT"
        fi
done
1 Like

Thank you so much. Much effort to get something most other tools can do by command-line options :slight_smile:
I have scheduled par2 for our existing restic repos for Friday, so it can finish over the weekend…

I am curious which other backup tools have error correction?

Duplicity is the only one I am familiar with. But it doesn’t do deduplication, making long chains fragile. Any other?

Indeed i had to restore a 8 years old version of file some weeks ago and was very happy not no have any errors on this 8 years more or less untouched backup (bacula)…