Sharing a convenience wrapper for local disk backups on home server

Hello,
I learned about restic last week and decided to do all backups with it on my home server. So, thank you for restic!

To simplify this process I created a bash script that wraps around restic and can be used in the scheduler of your choice. It takes care of things like preventing restic to run more than once and does housekeeping to remove old snapshots. If you like the description in the script then my recommendation to use it would be this:

  1. Edit the script and go to the configuration section. Update the variables source_path, repo_path, restore_path and repo_pw.
  2. Make sure all the folders exist (only restore_path folder is created if missing)
  3. Execute the script with “-init” to create your repository
  4. Execute the script with “-backup_dryrun” to understand if it does what it is supposed to do
  5. Call the script with no parameters or “-h” or “–help” to get help
  6. You may want to configure logging in the script at this point in time (go to configuration section)
  7. Play around with the restore options provided by the script

This script is for for simple use-cases with local disk interaction only. It can be a building block if you want to decouple the backup process from uploading your repository to a remote host with a separate script.

Hope this is useful for the community.

Here it is, I call it “Restic_local.bash” but you can change the name to whatever you want. The formatting does not work properly when pasting here, sorry for this..

#!/bin/bash

# First: Kudos to restic!


##### Use-case description #####
# Schedule hourly backups via this script wrapper on your home server.
# The script will determine if another restic instance runs and skips a run
# if this is the case. Old snapshots are removed as per policy configuration.
# The forget, prune and check actions are triggered as per user 
# configuration in this script and execute only once per day when due.


##### Detailed description #####
# This script is a convenience wrapper to automate local disk backups on your
#       home server.
# Configure this script in the next section and call it via cron or systemd.
#       The script will manage the rest for simple use-cases.
# Renaming the script is no problem, it will still work.
# The script exits immediately when another restic process is running. 
#       This way you don't need to worry too much about scheduling.
# Backup a single folder from and to a local disk 
#       is supported (no remote hosts).
# Restore into a predefined directory on local disk is supported.
# Tests if the restore directory exists and try to create it otherwise
# For automated scenarios, messages go to a log file if configured.
#       You can start without logging for an interactive experience and 
#       switch later. Watch out for lines containing the string "ERROR",
#       this indicates something needs your attention.
# Messages are send to stdout unless configured for logging.
# Sets the Pack Size to 64 MB.
# Applies --no-cache and when possible (this means restic version >= 0.18.0)
#       also --no-scan and --skip-if-unchanged on backups.
# Test if source and backup folder exist and exit immediately if not.
#       This is to protect new users of the script from modifying their
#       system without intend, e.g. on the first run.
# Repack configuration via --max-unused limit in percentage is 
#       possible - default 5%
# The script removes old snapshots based on user configured policy for weekly,
#       daily and hourly snapshots. The script takes care that forget, prune
#       and check calls are done only once a day when due. The minimum interval
#       between these clean ups is configured via the variable
#       delay_forget_and_prune_days. The minimum is 2.
#       A lock file under /tmp/restic_local.lock is used to sync this process.
# Tested with restic 0.18.0 compiled with go1.24.1 on linux/amd64.
# Tested with restic 0.14.0 compiled with go1.19.8 on linux/arm64.
#       -restore*clean calls fail with this version since --delete is not
#       there. --no-scan and --skip-if-unchanged unsupported for backup.
# BUGS
#	The log file grow forever.
#	The script exits if logging is configured but log file cant be created
#	No support for environment variables
#	No support for complex scenarios, e.g. remote hosts
#		This would require to enable caching.





##### User configuration - modify only after the "=" sign #####
source_path=~/Dokumente				# Local Folder to backup from
repo_path=~/repo				# Repository folder
restore_path=~/restore				# Restore into this folder
repo_pw="secret123"				# Encrypt with this string

# End of core configuration.

log_to_file=false		# If "true", log to file instead to stdout
logfile=$repo_path/log.txt	# Log-file to append messages to
delay_forget_and_prune_days=2	# Housekeeping every # days; 2 is minimum!
keep_weekly_snapshots_max=12	# Maximum number of weekly snapshots to keep
keep_daily_snapshots_max=14	# Maximum number of daily snapshots to keep
keep_hourly_snapshots_max=24	# Maximum number of hourly snapshots to keep
unused_space_max_percentage=5	# Restic default is 5%, max % for unused space


# End of user configuration.
# Do not modify anything from here unless you know what you're doing





##### List of exit codes #####
# 1 = No arguments given, printing help
# 2 = Unknown argument given, printing help
# 3 = Missing parameter to -find
# 4 = Cannot create restore dir
# 5 = The directory configured via $source_path or $repo_path does not exist
# 6 = Missing parameter to -restore_elements
# 7 = Cannot create logfile
# 8 = Ambiguous command to restore_snapshot
# 9 = Missing argument to restore_snapshot
# 10 = Found running instance of restic
# 11 = Failed to forget
# 12 = Failed to prune
# 13 = Repository is corrupt - that means restic check failed
# 14 = Version check for restic failed
# 15 = Restic version is too low to execute this



##### Environment variables for restic #####
export RESTIC_REPOSITORY=$repo_path
export RESTIC_PASSWORD=$repo_pw
export RESTIC_PACK_SIZE=64




###### Program variables ######
lock_file=/tmp/restic_local.lock  # Ensure housekeeping runs only once per day
use_all_features=0		  # Apply options based on restic version 			


###### Function definition ######

print_help () {
        echo "

        Usage:

        Please remember to edit the script and change the source_path,
        repo_path, repo_pw, restore_path and logging section accordingly.


        -init
                Initialize a new repository at $repo_path

        -backup
    		Start a backup from $source_path

        -backup_dryrun
    		Start a dry run backup

        -find <name>
    		Search for <name> in the repository, globbing is allowed but
    		requires quoting. E.g.: -find 'name*'

        -snapshots
    		Show snapshots

        -restore_snapshot <snapshot_id|latest>
		Full restore of snapshot_id or >latest< to $restore_path which
		is created if it does not exist.

        -restore_snapshot_clean <snapshot_id|latest>
		Same as before but adding the --delete option to allow
		a 1:1 restore. Requires restic version 0.18.0 or higher.

        -restore_elements <snapshot_id> <file/dir>
		Restore single files or folders from a specific snapshot.
		Use -find to get the <snapshot_id> and name of <file/dir> 
		to use here

        -restore_elements_clean <snapshot_id> <file/dir>
		Same as before but adding the --delete option from restic. 
		Requires restic version 0.18.0 or higher.
        "
}

init_log () {
	if [ $log_to_file == "true" ]; then
		touch $logfile
		if [ $? -ne 0 ]; then
			echo "Cannot create log file, exiting"
			exit 7
		fi
	fi
}

log () {
	message=$1
	if [ $log_to_file == "true" ]; then
		echo "$(date): $message" >> $logfile
	else
		echo "$(date): $message"
	fi
}

create_repo () {
        restic init
	return $?
}

init_repo () {
        log "INFO: Creating repository"
        create_repo
        status_repo_creation=$?

        if [ $status_repo_creation -eq 0 ]; then
                log "INFO: Successfully created repository"
        else
                log "ERROR: Failed to create repository, is there already one?"
                exit $status_repo_creation
        fi
}

parameter_version_check () {
        # Check if we run at least version 0.18.0 to be able to use the
        # --skip-if-unchanged switch. This wasnt working on 0.14.0.
        version_minor=$(restic version | awk '{print $2}' | \
                awk -F'.' '{print $2}')
        version_status=$?
        
        if [ $version_status -ne 0 ]; then
                log "ERROR: Version check failed. Got: $version_status by awk"
                exit 14
        else
                if (( $version_minor >= 18 )); then
                	log "INFO: Found restic 0.18.0 or higher."
                        return 0
                else
                	log "INFO: Found restic is lower than 0.18.0."
                        return 1
                fi
        fi
}

backup () {
        log "INFO: Starting backup"

        backup_status=255

        if [ $use_all_features -eq 0 ]; then
                restic backup . --no-scan --no-cache --skip-if-unchanged
                backup_status=$?
        else
                restic backup . --no-cache
                backup_status=$?
        fi

        if [ $backup_status -eq 0 ]; then
                log "INFO: Backup successfully completed"
        else
                log "ERROR: Backup not successful"
        fi
        
        housekeeping
}

backup_dryrun () {
        restic backup . --dry-run -vv --no-cache
        return $?
}

find_with_pattern () {
        search_pattern=$1
        if [ -z $search_pattern ]; then
                log "ERROR: Missing parameter to find, dont forget to quote"
                exit 3
        else
                restic find $search_pattern --no-cache
        fi
}

snapshots () {
	restic snapshots --no-cache
}

check_if_restore_dir_exists () {
	if [ ! -d $restore_path ]; then
		mkdir $restore_path > /dev/null 2>&1
		if [ $? -ne 0 ]; then
			log "ERROR: Cannot create the restore dir"
			exit 4
		fi
	fi
}


restore_snapshot () {
        snapshot_id=$1
        should_del=$2
        restore_status=255
        log "INFO: Restoring a snapshot"
        check_if_restore_dir_exists

        if [ -z $snapshot_id ]; then
	        log "ERROR: Missing argument to restore_snapshot"
	        exit 9
        fi


        if [ -z $should_del ]; then
	        log "INFO: Restoring snapshot with id: $snapshot_id"
	        restic restore $snapshot_id --target $restore_path --no-cache
	        restore_status=$?
        elif [ $should_del == "clean" ]; then
	        if [ $use_all_features -ne 0 ]; then
	        	log "ERROR: Cannot execute due to old restic version."
	        	log "ERROR: Please consider using no_clean option."
	        	exit 15
	        fi
		log "INFO: Restoring a clean snapshot with id: $snapshot_id"
		restic restore $snapshot_id --target $restore_path \
			--no-cache --delete
	        restore_status=$?
        else
                log "ERROR: Ambiguous command to restore_snapshot."
                exit 8
	fi

        if [ $restore_status -eq 0 ]; then
                log "INFO: Successfully restored snapshot"
        else
                log "ERROR: Failed to restore snapshot"
        fi
}

restore_elements () {
        log "INFO: Restoring single element"
        check_if_restore_dir_exists

        snapshot_id=$1
        file_dir=$2
        should_del=$3
        restore_status=255

        if [ -z $snapshot_id ]; then
                log "ERROR: Please provide a snapshot id"
                exit 6
        elif [ -z $file_dir ]; then
                log "ERROR: Please provide a file or directory name"
                exit 6
        elif [ -z $should_del ]; then
	        should_del="false"
        fi


        if [ $should_del == "clean" ]; then
 
		if [ $use_all_features -ne 0 ]; then
			log "ERROR: Cannot execute due to old restic version."
	        	log "ERROR: Please consider using no_clean option."
			exit 15
		fi
	        log "INFO: Clean restore of $file_dir at snapshot $snapshot_id"
	        restic restore $snapshot_id --include $file_dir --target \
	                $restore_path --no-cache --delete
	        restore_status=$?
        else
	        log "INFO: Restoring $file_dir from snapshot $snapshot_id"
	        restic restore $snapshot_id --include $file_dir --target \
	                $restore_path --no-cache
	        restore_status=$?
        fi

        if [ $restore_status -eq 0 ]; then
	        log "INFO: Restoring elements successful"
        else
	        log "ERROR: Failed to restore elements"
        fi
}


check_if_restic_runs_already () {
	my_name=$(basename $0)
	ps -ef | grep restic | grep -v grep | grep -v $my_name | \
	        grep restic > /dev/null 2>&1
	am_i_running=$?

	if [ $am_i_running -eq 0 ]; then
		log "ERROR: Detected a running instance of restic."
		log "ERROR: Skipping this run to avoid issues."
		exit 10
	fi
}


execute_forget_and_prune () {
        # We know now housekeeping is due but we don't know yet if we already 
        # ran on this day. Only one run per day should happen regardless how
        # often the script is called. Avoid waste of electricity.

        if [ ! -f $lock_file ] ; then
                # First run for today, create the lock file to avoid duplicate
                # executions per day. The lock file is removed by the 
                # housekeeping function when appropriate.
                log "INFO: Housekeeping started.."

                touch $lock_file

                restic forget --keep-weekly $keep_weekly_snapshots_max \
                        --keep-daily $keep_daily_snapshots_max \
                        --keep-hourly $keep_hourly_snapshots_max --no-cache
                if [ $? -ne 0 ]; then
                        log "ERROR: Failed to forget snapshots as per policy."
                        log "ERROR: Please take care of the repository"
                        exit 11
                fi

                restic prune --max-unused ${unused_space_max_percentage}% \
                        --no-cache
                if [ $? -ne 0 ]; then
                        log "ERROR: Failed to prune snapshots as per policy."
                        log "ERROR: Please take care of the repository"
                        exit 12
                fi

                restic check --no-cache
                if [ $? -ne 0 ]; then
                        log "ERROR: Repository is corrupt, please check it."
                        exit 13
                fi
                
                log "INFO: Housekeeping completed successfully."
                log "INFO: Skipping other attempts for today."
        fi
}

housekeeping () {
        # Check if housekeeping is due
        day_of_the_year=$(date +%j)
        is_zero=$(($day_of_the_year % $delay_forget_and_prune_days))

        if [ $is_zero -ne 0 ]; then
                # No housekeeping required, therefore we don't log to prevent
                # flooding it. We delete the lock file if it exists to signal
                # that housekeeping period has been left. A new one is run
                # when required.
                rm $lock_file > /dev/null 2>&1
                return
        else
                execute_forget_and_prune
        fi
}




##### Main ######



# Init the log if required
init_log



# Check if another instance of restic is running to prevent potential issues
# with automation and parallel restic sessions, see
# https://restic.readthedocs.io/en/stable/040_backup.html#scheduling-backups
check_if_restic_runs_already




# Check what version of restic is available to understand what options to use
parameter_version_check
use_all_features=$?


# A user may not configure the script on the first run, so we want to exit 
# if $source_path or $repo_path dont exist to avoid unwanted change
if [ ! -d $source_path ]; then
	log "ERROR: Please point source_path variable to an existing directory"
	exit 5
elif [ ! -d $repo_path ]; then
    	log "ERROR: Please point repo_path variable to an existing directory"
	exit 5
fi



# needed for --skip-if-unchanged as per
# https://restic.readthedocs.io/en/stable/040_backup.html#skip-creating-snapshots-if-unchanged
cd $source_path




# Check parameters
if [ -z $1 ]; then
        print_help
        exit 1
elif [ $1 == "-init" ]; then
        init_repo
elif [ $1 == "-backup" ]; then
        backup
elif [ $1 == "-backup_dryrun" ]; then
        backup_dryrun
elif [ $1 == "-find" ]; then
        find_with_pattern $2
elif [ $1 == "-snapshots" ]; then
        snapshots
elif [ $1 == "-restore_snapshot" ]; then
        restore_snapshot $2
elif [ $1 == "-restore_snapshot_clean" ]; then
        restore_snapshot $2 "clean"
elif [ $1 == "-restore_elements" ]; then
        restore_elements $2 $3
elif [ $1 == "-restore_elements_clean" ]; then
        restore_elements $2 $3 "clean"
elif [ $1 == "-h" ]; then
        print_help
elif [ $1 == "--help" ]; then
        print_help
else
        echo
        log "ERROR: Unknown command: $1"
        print_help
        exit 2
fi

There was a typo in the description of error code 8 in the comments which is corrected now