No, you can’t do that. The second line will simply replace the value in that variable, as if the first line was never there.
Nor can you name two paths in the same variable, because that’s not how restic is expecting the repository to be specified (it only expects one repository to be listed).
Simply move the RESTIC_PASSWORD line to the top, then duplicate the three other lines and adjust their contents accordingly.
That said, I’m not sure it’s a great idea to prune every time you have run a backup. Seems a bit excessive to me.
No, just do what I said within the same script. The script is just there to run your commands, so unless you have specific needs you can keep them all in one script.
Because it makes the code cleaner that way. It makes no practical difference.
OK, I did that; now get Fatal: unable to open repo at /media/…/Backup: ReadDir: open /media/…/Backup/keys: permission denied
There’s a key in the keys folder though
But keep in mind that if you use a loop like this, your data could change in the time the first backup starts and finishes when coming to the second repository. If that is something you can live with, carry on If not it would make sense to use some sort of snapshot like in BTRFS and the likes.
Note that this applies whether you are running two consecutive commands or using a loop (which is just decoration for consecutive commands).
Using snapshots is preferred for other reasons as well, such as making sure a single snapshot is internally consistent. For example, if directly backing up a database server’s data directory, you must backup a snapshot (on stop the server) or the backed-up data will almost certainly be corrupt. We use LVM snapshots on our servers to take backups of running services without interruption.
If anyone’s interested, assuming you have enough resources (Memory + CPU) and you have plenty of throughput to/from your storage, we can expand on @cdhowie 's idea a little to make the backups run in parallel.
This assumes that you have your restic password exported somewhere in the file, and that it’s the same for each remote. It can be modified to work around this. (Ask me if you want help with that. If you do, please let me know if you want to export the passwords in the file or prompt for them.) It also still requires each remote to not contain any spaces.
Here’s the basic idea (Based on cdhowie’s idea):
#!/bin/bash
# The Local Directory to Backup. CHANGE THIS TO WHATEVER YOU WANT TO BACKUP!
export LOCAL_DIR="~/whatnot" #CHANGE ME!
# REPOS - The Repositories to backup to, separated by spaces. Remote names can't have a space in them.
# CHANGE THIS TO WHATEVER REMOTES YOU WANT!
export REPOS="remote1:directory remote2:directory2" # CHANGE ME!
export PROCS='' # DON'T CHANGE! - Needed to Sync the threads at the end
# Function backup_one backs up the location "$LOCAL_DIR" to the remote given as the first argument.
# The code below will call this automatically
backup_one() {
/usr/local/bin/restic -r "$1" backup "${LOCAL_DIR}"
/bin/sync
/usr/local/bin/restic -r "$1" forget --keep-last 3 --prune
}
# Loop over the list of repos, spawning a new thread for each one
for repo in $REPOS; do
backup_one "$repo" &
export PROCS="$PROCS $!"
done
for proc in $PROCS; do
wait $proc
done
Hopefully this helps someone.
Good luck all,
jedi453
If I’m not mistaken, you can replace this with a single line: wait (and also remove the $PROCS variable). wait by itself should wait for all background jobs.
Side note, you don’t need to export any of these variables. Exporting a variable adds it to the environment of commands that are executed within the shell, but none of these variables are used outside of this shell script.
We can simplify the script:
#!/bin/bash
# The Local Directory to Backup. CHANGE THIS TO WHATEVER YOU WANT TO BACKUP!
LOCAL_DIR="~/whatnot" #CHANGE ME!
# REPOS - The Repositories to backup to, separated by spaces. Remote names
# can't have a space in them.
#
# CHANGE THIS TO WHATEVER REMOTES YOU WANT!
REPOS="remote1:directory remote2:directory2" # CHANGE ME!
# Function backup_one backs up the location "$LOCAL_DIR" to the remote given as
# the first argument. The code below will call this automatically.
backup_one() {
/usr/local/bin/restic -r "$1" backup "$LOCAL_DIR"
/bin/sync
/usr/local/bin/restic -r "$1" forget --keep-last 3 --prune
}
# Loop over the list of repos, running each backup in the background.
for repo in $REPOS; do
backup_one "$repo" &
done
# Wait for all backups to complete.
wait
I unset my variables after the processes are done.
$ unset THE_ENV_VAR1 THE_ENV_VAR2
It could also be used to set the environment variable again inside the script.
$ RESTIC_REPOSITORY="/thatplace"
$ restic …
$ unset RESTIC_REPOSITORY
$ restic …
$ RESTIC_REPOSITORY=“thatotherplace”
$ unset RESTIC_REPOSITORY
`I store my password inside a gpg file that only my user (and root) can access.
#!/usr/bin/env bash
#
set -o errexit
set -o errtrace
set -o nounset
set -o pipefail
#set -o xtrace
#
# functions
function main() {
# Elegant exit
trap exitStageLeft EXIT ERR
# Backup and Prune them
backupAndPrune
}
function exitStageLeft() {
# Unset the variables after everything is done
printf "\n%s\n" "Cleaning up before leaving!"
unset -f backupAndPrune
}
function backupAndPrune() {
# export can be used instead of local, but it let the variable available for subshells
# The environment variables for restic
#local RESTIC_REPOSITORY
#local RESTIC_PASSWORD
#local RESTIC_PASSWORD_FILE
local RESTIC_PASSWORD_COMMAND
#local AWS_ACCESS_KEY_ID="super_hash"
#local AWS_SECRET_ACCESS_KEY="hyper_hash"
#local PUBLIC_ENDPOINT_LOCATION="http://someplaceontheinter.webs"
# Custom variables
#local RESTIC_REMOTE_REPO
#local RESTIC_EXCLUDED
#local RESTIC_INCLUDED
local RESTIC_SOURCE
#local RESTIC_USER_HOME
local RESTIC_BIN
declare -a RESTIC_REPOS # declared as an array
#
#local RESTIC_PASSWORD=
RESTIC_BIN="$(\which restic)"
RESTIC_REPOS=("/…/USB-Backup/backup" "/…/2ndDIR/backup")
RESTIC_SOURCE="${HOME}/"
RESTIC_PASSWORD_COMMAND="\gpg --quiet --decrypt /home/${USER:-}/DoNotTouch/pwd.restic.txt.gpg"
# Do the backups first
printf "\n%s\n" "Backing up first"
for repo in "${RESTIC_REPOS[@]}"; do
"${RESTIC_BIN}" --verbose --repo "${repo}" backup "${RESTIC_SOURCE}"
done
printf "\n%s\n" "Pruning after the backups"
for repo in "${RESTIC_REPOS[@]}"; do
"${RESTIC_BIN}" --verbose --repo "${repo}" forget --keep-last 3 --prune
done
}
main "${@}"
If not it would make sense to use some sort of snapshot like in BTRFS and the likes.