How to create par2 parity files for a Restic repository

1

I would like to generate parchive par2 parity files for long-term storage of a restic repository. This helps to recover from errors in case repository is corrupted. Restic currently doesn’t have error correction feature (Duplicity is the only backup app that generates par2 files AFAIK).

Does anyone know what’s the best way to create these par2 files? Should I write a bash script to recursively create par2 files for each file in the restic depository? Or should I generate a tar file from the whole restic repository and then create Par2 files for that one file? But this increases the required space by factor 2 since tarball must be saved too (a tarball generated 10 years later from the same repository may not be same as today’s tarball due to software updates).

Anyone has dealt with error correction with Restic?

3 Likes
2

That’s the way I do it. It means that it’s nice and quick to make par2 files for the newest additions after each backup (just find any recently-created files). I don’t tend to prune much, so am happy for the odd par2 file to hang around even when the corresponding restic file has been deleted. Otherwise, just make a script to delete the unused par2 files after a prune operation.

In order to avoid making a mess in my repo directory (although I don’t think restic would care if I did), I have a corresponding directory tree just for the par2 files.

My scripts are rather amateurish, and proper engineers would probably balk at them. Happy to share if needed, but it would be better to have more elegant examples in the forum if anyone has any? :slight_smile:

2 Likes
3

How do you convince par2create to write the par2 files to a different location? My par2 (debian10, 0.8.0) ignores my different name/location :frowning:

4

Yes, I seem to recall that was a bit of a hassle. I think you have to be careful from where you execute the command. My script does the following (I don’t know which bits are critical, and which are just how I wrote it)…
Assuming you want to make a parchive of the file:
/backup/restic_repo/data/c4/c4048768c4bd77232a7ffc5c4c3671b13e0234c246c80ceabbcd1a68365d1c58
into the file:
/backup/restic_par/restic_repo/data/c4/c4048768c4bd77232a7ffc5c4c3671b13e0234c246c80ceabbcd1a68365d1c58-5.par2

Then run the following commands:

cd /backup
par2create -B ./ -p -n1 -r5 /backup/restic-par/restic-repo/data/c4/c4048768c4bd77232a7ffc5c4c3671b13e0234c246c80ceabbcd1a68365d1c58 restic-repo/data/c4/c4048768c4bd77232a7ffc5c4c3671b13e0234c246c80ceabbcd1a68365d1c58
My full script is below. Use at your own risk :) 
#!/bin/bash

#paths to the repo and the parchive directory
rootpath=/path/below/repo
repodir=restic_repo
pardir=restic_repo_parchive

#percentage redundancy desired
PERC=5

#oldest file for which to create parchive files (-3 = 3 days before present)
mtime=-3

cd $rootpath
#find any files changed within $mtime days and pipe them into the do loop 
find $repodir -type f -mtime $mtime -print0 | while IFS= read -r -d '' FILE
do
        DIR=$(dirname "${FILE}")
        DIR_OUT="$rootpath/$pardir/${DIR}"
        FILENAME=$(basename "${FILE}")
        PAR_ROOT_OUT="$DIR_OUT/${FILENAME}"
        FILE_OUT="$DIR_OUT/${FILENAME}"-$PERC.par2
#create directory if it doesn't exist
        if [[ ! -d $DIR_OUT ]]; then
                echo making directory "$DIR_OUT"
                mkdir -p "$DIR_OUT"
        fi 
#create par file if it doesn't exist
        if [ ! -f "$FILE_OUT" ]; then
                echo executing: par2create -B ./ -p -n1 -r$PERC "${PAR_ROOT_OUT}" "${FILE}"
                echo in `pwd`
                par2create -B ./ -q -n1 -r$PERC "${PAR_ROOT_OUT}" "${FILE}"
#remove the unnecessary file created by par2create
                rm "${PAR_ROOT_OUT}".par2
#and rename the other file to the correct name
                mv "${PAR_ROOT_OUT}".vol*par2 "$FILE_OUT"
        fi
done
1 Like
5

Thank you so much. Much effort to get something most other tools can do by command-line options :slight_smile:
I have scheduled par2 for our existing restic repos for Friday, so it can finish over the weekend…

6

I am curious which other backup tools have error correction?

Duplicity is the only one I am familiar with. But it doesn’t do deduplication, making long chains fragile. Any other?

7

Indeed i had to restore a 8 years old version of file some weeks ago and was very happy not no have any errors on this 8 years more or less untouched backup (bacula)…

8

We just update our repos to compressed v2-repos so we have a lot of orphan par2-files.

So here our modification to delete orphan par2-files:

#!/bin/bash

## https://forum.restic.net/t/how-to-create-par2-parity-files-for-a-restic-repository/5149/4 ##

# paths to the repo and the parchive directory
rootpath=/mnt/nas-2/restic-backup
  [ -d $rootpath ] || exit 1
  cd   $rootpath   || exit 1
        #ls -l $rootpath
        #       drwxr-xr-x  3 root     root 4096 Jun 29  2022 repo-par2
        #       drwxrwx--- 38 www-data root 4096 Nov 29 15:38 rest-server

repodir_base=rest-server
pardir=repo-par2

# one repo per server ($1)
repodir=$repodir_base/$1
  [ "$1" = "" ]   && exit 1
  [ -d $repodir ] || exit 1

pardir_cleanup=$pardir/$repodir_base/$1
  [ "$1" = "" ]   && exit 1
  [ -d $repodir ] || exit 1

# percentage redundancy desired
  PERC=5

# oldest file for which to create parchive files (-3 = 3 days before present)
  # mtime=-3000 # initial par2
  mtime=-7 # initial par2

##############
## cleanup: ##
##############
if [ "$2" = "cleanup" ] || [ "$3" = "cleanup" ] ; then
  find $pardir_cleanup -type f -name *.par2 -print0 | while IFS= read -r -d '' FILE
  do
        DIR=$(dirname "${FILE}")
        DIR_IN=$(echo "$DIR" | sed -r "s!^$pardir/!!")
        FILENAME=$(basename "${FILE}")
                FILENAME_IN=$(echo "$FILENAME" | sed -r 's/\-[0-9]+\.par2$//')
        FILE_IN="$DIR_IN/${FILENAME_IN}"
                [ "$2" = "debug" ] && echo
                [ "$2" = "debug" ] && echo "DIR:    $DIR"
                [ "$2" = "debug" ] && echo "FILE:   $FILE"
                [ "$2" = "debug" ] && echo "FILE_IN:          $FILE_IN"
        if [ -f "$FILE_IN" ] ; then
                [ "$2" = "debug" ] && echo "keep:   $FILE"
        else
                [ "$2" = "debug" ] && echo "orphan: $FILE  (=> delete)"
                rm -f "$FILE"
        fi
  done
fi ## cleanup

#  find any files changed within $mtime days and pipe them into the do loop
find $repodir -type f -mtime $mtime -print0 | while IFS= read -r -d '' FILE
do
        DIR=$(dirname "${FILE}")
        DIR_OUT="$rootpath/$pardir/${DIR}"
        FILENAME=$(basename "${FILE}")
        PAR_ROOT_OUT="$DIR_OUT/${FILENAME}"
        FILE_OUT="$DIR_OUT/${FILENAME}"-$PERC.par2
        # create directory if it doesn't exist
        if [[ ! -d $DIR_OUT ]]; then
                [ "$2" = "debug" ] && echo mkdir "$DIR_OUT"
                mkdir -p "$DIR_OUT"
        fi
        # create par file if it doesn't exist
        if [ ! -f "$FILE_OUT" ]; then
                [ "$2" = "debug" ] && echo
                [ "$2" = "debug" ] && echo executing: par2create -B ./ -p -n1 -r$PERC "${PAR_ROOT_OUT}" "${FILE}"
                [ "$2" = "debug" ] && echo in `pwd`
                par2create -B ./ -qq -n1 -r$PERC "${PAR_ROOT_OUT}" "${FILE}" || exit 1
                [ "$2" = "debug" ] && ls -lh "${PAR_ROOT_OUT}".*par2
                # remove the unnecessary file created by par2create
                  rm "${PAR_ROOT_OUT}".par2
                # and rename the other file to the correct name
                  mv "${PAR_ROOT_OUT}".vol*par2 "$FILE_OUT"
        fi
done


exit 0
2 Likes
9

Actually a much nicer solution would be to post-process created/deleted files directly after they are written to or deleted from the backend. That would need support from restic, but it would be only a comparably small enhancement.

I think about implementing such a feature in rustic.

10

FYI, this feature is now available in the latest rustic beta. For an example how to configure rustic to generate (and remove) .par2 files, see https://github.com/rustic-rs/rustic/blob/main/examples/par2.toml

2 Likes
11

This is great! Amazing how fast and easily you can push forward Rustic.

I suppose it can add par2 files to the current restic repositories. Will Restic normally function afterwards, since par2 files are separate? I have a lot of backup scripts with restic.

12

Sorry that I have to disappoint you - this does only add par2 files to newly added files (and remove the corresponding par2 files when files are removed). So, for existing repos the par2 files must be manually generated for all already existing repo files.

restic (and rustic) will happily ignore additional files which don’t have a filename corresponding to a sha256 hash. But keep in mind that restic won’t add .par2 files to newly created files unless a similar solution is added like the one I’ve added to rustic.

1 Like