Hello,
I learned about restic last week and decided to do all backups with it on my home server. So, thank you for restic!
To simplify this process I created a bash script that wraps around restic and can be used in the scheduler of your choice. It takes care of things like preventing restic to run more than once and does housekeeping to remove old snapshots. If you like the description in the script then my recommendation to use it would be this:
- Edit the script and go to the configuration section. Update the variables source_path, repo_path, restore_path and repo_pw.
- Make sure all the folders exist (only restore_path folder is created if missing)
- Execute the script with “-init” to create your repository
- Execute the script with “-backup_dryrun” to understand if it does what it is supposed to do
- Call the script with no parameters or “-h” or “–help” to get help
- You may want to configure logging in the script at this point in time (go to configuration section)
- Play around with the restore options provided by the script
This script is for for simple use-cases with local disk interaction only. It can be a building block if you want to decouple the backup process from uploading your repository to a remote host with a separate script.
Hope this is useful for the community.
Here it is, I call it “Restic_local.bash” but you can change the name to whatever you want. The formatting does not work properly when pasting here, sorry for this..
#!/bin/bash
# First: Kudos to restic!
##### Use-case description #####
# Schedule hourly backups via this script wrapper on your home server.
# The script will determine if another restic instance runs and skips a run
# if this is the case. Old snapshots are removed as per policy configuration.
# The forget, prune and check actions are triggered as per user
# configuration in this script and execute only once per day when due.
##### Detailed description #####
# This script is a convenience wrapper to automate local disk backups on your
# home server.
# Configure this script in the next section and call it via cron or systemd.
# The script will manage the rest for simple use-cases.
# Renaming the script is no problem, it will still work.
# The script exits immediately when another restic process is running.
# This way you don't need to worry too much about scheduling.
# Backup a single folder from and to a local disk
# is supported (no remote hosts).
# Restore into a predefined directory on local disk is supported.
# Tests if the restore directory exists and try to create it otherwise
# For automated scenarios, messages go to a log file if configured.
# You can start without logging for an interactive experience and
# switch later. Watch out for lines containing the string "ERROR",
# this indicates something needs your attention.
# Messages are send to stdout unless configured for logging.
# Sets the Pack Size to 64 MB.
# Applies --no-cache and when possible (this means restic version >= 0.18.0)
# also --no-scan and --skip-if-unchanged on backups.
# Test if source and backup folder exist and exit immediately if not.
# This is to protect new users of the script from modifying their
# system without intend, e.g. on the first run.
# Repack configuration via --max-unused limit in percentage is
# possible - default 5%
# The script removes old snapshots based on user configured policy for weekly,
# daily and hourly snapshots. The script takes care that forget, prune
# and check calls are done only once a day when due. The minimum interval
# between these clean ups is configured via the variable
# delay_forget_and_prune_days. The minimum is 2.
# A lock file under /tmp/restic_local.lock is used to sync this process.
# Tested with restic 0.18.0 compiled with go1.24.1 on linux/amd64.
# Tested with restic 0.14.0 compiled with go1.19.8 on linux/arm64.
# -restore*clean calls fail with this version since --delete is not
# there. --no-scan and --skip-if-unchanged unsupported for backup.
# BUGS
# The log file grow forever.
# The script exits if logging is configured but log file cant be created
# No support for environment variables
# No support for complex scenarios, e.g. remote hosts
# This would require to enable caching.
##### User configuration - modify only after the "=" sign #####
source_path=~/Dokumente # Local Folder to backup from
repo_path=~/repo # Repository folder
restore_path=~/restore # Restore into this folder
repo_pw="secret123" # Encrypt with this string
# End of core configuration.
log_to_file=false # If "true", log to file instead to stdout
logfile=$repo_path/log.txt # Log-file to append messages to
delay_forget_and_prune_days=2 # Housekeeping every # days; 2 is minimum!
keep_weekly_snapshots_max=12 # Maximum number of weekly snapshots to keep
keep_daily_snapshots_max=14 # Maximum number of daily snapshots to keep
keep_hourly_snapshots_max=24 # Maximum number of hourly snapshots to keep
unused_space_max_percentage=5 # Restic default is 5%, max % for unused space
# End of user configuration.
# Do not modify anything from here unless you know what you're doing
##### List of exit codes #####
# 1 = No arguments given, printing help
# 2 = Unknown argument given, printing help
# 3 = Missing parameter to -find
# 4 = Cannot create restore dir
# 5 = The directory configured via $source_path or $repo_path does not exist
# 6 = Missing parameter to -restore_elements
# 7 = Cannot create logfile
# 8 = Ambiguous command to restore_snapshot
# 9 = Missing argument to restore_snapshot
# 10 = Found running instance of restic
# 11 = Failed to forget
# 12 = Failed to prune
# 13 = Repository is corrupt - that means restic check failed
# 14 = Version check for restic failed
# 15 = Restic version is too low to execute this
##### Environment variables for restic #####
export RESTIC_REPOSITORY=$repo_path
export RESTIC_PASSWORD=$repo_pw
export RESTIC_PACK_SIZE=64
###### Program variables ######
lock_file=/tmp/restic_local.lock # Ensure housekeeping runs only once per day
use_all_features=0 # Apply options based on restic version
###### Function definition ######
print_help () {
echo "
Usage:
Please remember to edit the script and change the source_path,
repo_path, repo_pw, restore_path and logging section accordingly.
-init
Initialize a new repository at $repo_path
-backup
Start a backup from $source_path
-backup_dryrun
Start a dry run backup
-find <name>
Search for <name> in the repository, globbing is allowed but
requires quoting. E.g.: -find 'name*'
-snapshots
Show snapshots
-restore_snapshot <snapshot_id|latest>
Full restore of snapshot_id or >latest< to $restore_path which
is created if it does not exist.
-restore_snapshot_clean <snapshot_id|latest>
Same as before but adding the --delete option to allow
a 1:1 restore. Requires restic version 0.18.0 or higher.
-restore_elements <snapshot_id> <file/dir>
Restore single files or folders from a specific snapshot.
Use -find to get the <snapshot_id> and name of <file/dir>
to use here
-restore_elements_clean <snapshot_id> <file/dir>
Same as before but adding the --delete option from restic.
Requires restic version 0.18.0 or higher.
"
}
init_log () {
if [ $log_to_file == "true" ]; then
touch $logfile
if [ $? -ne 0 ]; then
echo "Cannot create log file, exiting"
exit 7
fi
fi
}
log () {
message=$1
if [ $log_to_file == "true" ]; then
echo "$(date): $message" >> $logfile
else
echo "$(date): $message"
fi
}
create_repo () {
restic init
return $?
}
init_repo () {
log "INFO: Creating repository"
create_repo
status_repo_creation=$?
if [ $status_repo_creation -eq 0 ]; then
log "INFO: Successfully created repository"
else
log "ERROR: Failed to create repository, is there already one?"
exit $status_repo_creation
fi
}
parameter_version_check () {
# Check if we run at least version 0.18.0 to be able to use the
# --skip-if-unchanged switch. This wasnt working on 0.14.0.
version_minor=$(restic version | awk '{print $2}' | \
awk -F'.' '{print $2}')
version_status=$?
if [ $version_status -ne 0 ]; then
log "ERROR: Version check failed. Got: $version_status by awk"
exit 14
else
if (( $version_minor >= 18 )); then
log "INFO: Found restic 0.18.0 or higher."
return 0
else
log "INFO: Found restic is lower than 0.18.0."
return 1
fi
fi
}
backup () {
log "INFO: Starting backup"
backup_status=255
if [ $use_all_features -eq 0 ]; then
restic backup . --no-scan --no-cache --skip-if-unchanged
backup_status=$?
else
restic backup . --no-cache
backup_status=$?
fi
if [ $backup_status -eq 0 ]; then
log "INFO: Backup successfully completed"
else
log "ERROR: Backup not successful"
fi
housekeeping
}
backup_dryrun () {
restic backup . --dry-run -vv --no-cache
return $?
}
find_with_pattern () {
search_pattern=$1
if [ -z $search_pattern ]; then
log "ERROR: Missing parameter to find, dont forget to quote"
exit 3
else
restic find $search_pattern --no-cache
fi
}
snapshots () {
restic snapshots --no-cache
}
check_if_restore_dir_exists () {
if [ ! -d $restore_path ]; then
mkdir $restore_path > /dev/null 2>&1
if [ $? -ne 0 ]; then
log "ERROR: Cannot create the restore dir"
exit 4
fi
fi
}
restore_snapshot () {
snapshot_id=$1
should_del=$2
restore_status=255
log "INFO: Restoring a snapshot"
check_if_restore_dir_exists
if [ -z $snapshot_id ]; then
log "ERROR: Missing argument to restore_snapshot"
exit 9
fi
if [ -z $should_del ]; then
log "INFO: Restoring snapshot with id: $snapshot_id"
restic restore $snapshot_id --target $restore_path --no-cache
restore_status=$?
elif [ $should_del == "clean" ]; then
if [ $use_all_features -ne 0 ]; then
log "ERROR: Cannot execute due to old restic version."
log "ERROR: Please consider using no_clean option."
exit 15
fi
log "INFO: Restoring a clean snapshot with id: $snapshot_id"
restic restore $snapshot_id --target $restore_path \
--no-cache --delete
restore_status=$?
else
log "ERROR: Ambiguous command to restore_snapshot."
exit 8
fi
if [ $restore_status -eq 0 ]; then
log "INFO: Successfully restored snapshot"
else
log "ERROR: Failed to restore snapshot"
fi
}
restore_elements () {
log "INFO: Restoring single element"
check_if_restore_dir_exists
snapshot_id=$1
file_dir=$2
should_del=$3
restore_status=255
if [ -z $snapshot_id ]; then
log "ERROR: Please provide a snapshot id"
exit 6
elif [ -z $file_dir ]; then
log "ERROR: Please provide a file or directory name"
exit 6
elif [ -z $should_del ]; then
should_del="false"
fi
if [ $should_del == "clean" ]; then
if [ $use_all_features -ne 0 ]; then
log "ERROR: Cannot execute due to old restic version."
log "ERROR: Please consider using no_clean option."
exit 15
fi
log "INFO: Clean restore of $file_dir at snapshot $snapshot_id"
restic restore $snapshot_id --include $file_dir --target \
$restore_path --no-cache --delete
restore_status=$?
else
log "INFO: Restoring $file_dir from snapshot $snapshot_id"
restic restore $snapshot_id --include $file_dir --target \
$restore_path --no-cache
restore_status=$?
fi
if [ $restore_status -eq 0 ]; then
log "INFO: Restoring elements successful"
else
log "ERROR: Failed to restore elements"
fi
}
check_if_restic_runs_already () {
my_name=$(basename $0)
ps -ef | grep restic | grep -v grep | grep -v $my_name | \
grep restic > /dev/null 2>&1
am_i_running=$?
if [ $am_i_running -eq 0 ]; then
log "ERROR: Detected a running instance of restic."
log "ERROR: Skipping this run to avoid issues."
exit 10
fi
}
execute_forget_and_prune () {
# We know now housekeeping is due but we don't know yet if we already
# ran on this day. Only one run per day should happen regardless how
# often the script is called. Avoid waste of electricity.
if [ ! -f $lock_file ] ; then
# First run for today, create the lock file to avoid duplicate
# executions per day. The lock file is removed by the
# housekeeping function when appropriate.
log "INFO: Housekeeping started.."
touch $lock_file
restic forget --keep-weekly $keep_weekly_snapshots_max \
--keep-daily $keep_daily_snapshots_max \
--keep-hourly $keep_hourly_snapshots_max --no-cache
if [ $? -ne 0 ]; then
log "ERROR: Failed to forget snapshots as per policy."
log "ERROR: Please take care of the repository"
exit 11
fi
restic prune --max-unused ${unused_space_max_percentage}% \
--no-cache
if [ $? -ne 0 ]; then
log "ERROR: Failed to prune snapshots as per policy."
log "ERROR: Please take care of the repository"
exit 12
fi
restic check --no-cache
if [ $? -ne 0 ]; then
log "ERROR: Repository is corrupt, please check it."
exit 13
fi
log "INFO: Housekeeping completed successfully."
log "INFO: Skipping other attempts for today."
fi
}
housekeeping () {
# Check if housekeeping is due
day_of_the_year=$(date +%j)
is_zero=$(($day_of_the_year % $delay_forget_and_prune_days))
if [ $is_zero -ne 0 ]; then
# No housekeeping required, therefore we don't log to prevent
# flooding it. We delete the lock file if it exists to signal
# that housekeeping period has been left. A new one is run
# when required.
rm $lock_file > /dev/null 2>&1
return
else
execute_forget_and_prune
fi
}
##### Main ######
# Init the log if required
init_log
# Check if another instance of restic is running to prevent potential issues
# with automation and parallel restic sessions, see
# https://restic.readthedocs.io/en/stable/040_backup.html#scheduling-backups
check_if_restic_runs_already
# Check what version of restic is available to understand what options to use
parameter_version_check
use_all_features=$?
# A user may not configure the script on the first run, so we want to exit
# if $source_path or $repo_path dont exist to avoid unwanted change
if [ ! -d $source_path ]; then
log "ERROR: Please point source_path variable to an existing directory"
exit 5
elif [ ! -d $repo_path ]; then
log "ERROR: Please point repo_path variable to an existing directory"
exit 5
fi
# needed for --skip-if-unchanged as per
# https://restic.readthedocs.io/en/stable/040_backup.html#skip-creating-snapshots-if-unchanged
cd $source_path
# Check parameters
if [ -z $1 ]; then
print_help
exit 1
elif [ $1 == "-init" ]; then
init_repo
elif [ $1 == "-backup" ]; then
backup
elif [ $1 == "-backup_dryrun" ]; then
backup_dryrun
elif [ $1 == "-find" ]; then
find_with_pattern $2
elif [ $1 == "-snapshots" ]; then
snapshots
elif [ $1 == "-restore_snapshot" ]; then
restore_snapshot $2
elif [ $1 == "-restore_snapshot_clean" ]; then
restore_snapshot $2 "clean"
elif [ $1 == "-restore_elements" ]; then
restore_elements $2 $3
elif [ $1 == "-restore_elements_clean" ]; then
restore_elements $2 $3 "clean"
elif [ $1 == "-h" ]; then
print_help
elif [ $1 == "--help" ]; then
print_help
else
echo
log "ERROR: Unknown command: $1"
print_help
exit 2
fi