Script works manually but not on Cron

Getting Help
1

Hi all:

First… thanks for your support and your excellent Restic tool.

I’m doing backups with a bash script that works fine when it’s launched manually, but not when it’s under Cron. I already fixed some issues related to permissions on OSx / Cron and so on… but still I have some fatal errors on Restic.

This is my environment:

Restic:
restic 0.12.1 compiled with go1.16.6 on darwin/amd64

OS:
ProductName: Mac OS X
ProductVersion: 10.15.7
BuildVersion: 19H1323

Backup to Google Drive via Rclone (works fine… I already tested).

This is my Script

#!/bin/sh

#Variables

now=$(date +%s)

start=$(date +%s)

#Cleaning Backups

find ~/Backup -name ‘.DS_Store’ -type f -delete > /dev/null 2>&1

#Cleaninig Restic Snapshots

/usr/local/bin/restic cache --cleanup > /dev/null 2>&1

echo “restic outputs for <>” $? >> ~/System_Scripts/error.log

##Forget old snapshots

/usr/local/bin/restic --password-file ~/.pera -r rclone:gdrive:1 forget --keep-last 30 --prune > /dev/null 2>&1

echo “restic outputs for <>” $? >> ~/System_Scripts/error.log

##Doing backups

#Backup Borg

export BORG_PASSPHRASE=$(cat ~/.melocoton)

/usr/local/bin/borg create ~/Backup_enc::$(date ‘+%A-%d-%B-%y-%T’) ~/1 --show-rc --compression auto,lzma,6 2>> ~/System_Scripts/error.log

export BORG_PASSPHRASE=""

#Backup Restic

/usr/local/bin/restic --password-file ~/.pera -r rclone:gdrive:1 backup ~/Backup_enc/ >> ~/System_Scripts/error.log

echo “restic outputs for <>” $? >> ~/System_Scripts/error.log

#Backup Output

finish=$(date +%s)

elapse=$((finish - start))

printf “\n >>>>>>>>> Executed in $elapse secs at $(date). Done Backup”

printf “\n >>>>>>>>> Executed in $elapse secs at $(date). Done Backup” >> ~/System_Scripts/backup.log

I launched backup.sh manually (the first time), after… cron execs it.

This is the backup log:

Executed in 35 secs at Sat Sep 4 11:04:21 CEST 2021. Done Backup
Executed in 7 secs at Sat Sep 4 11:15:07 CEST 2021. Done Backup
Executed in 7 secs at Sat Sep 4 11:30:07 CEST 2021. Done Backup
Executed in 6 secs at Sat Sep 4 11:45:06 CEST 2021. Done Backup

(the first one works… 35 secs …all good) The others did not.

This is the error.log:

restic outputs for <> 0
restic outputs for <> 0
terminating with success status, rc 0
using parent snapshot adfe8076

Files: 8 new, 1 changed, 509 unmodified
Dirs: 1 new, 5 changed, 0 unmodified
Added to the repo: 1.842 MiB

processed 518 files, 5.427 GiB in 0:13
snapshot b90504b9 saved
restic outputs for <> 0
restic outputs for <> 0
restic outputs for <> 1
terminating with success status, rc 0
restic outputs for <> 1
restic outputs for <> 0
restic outputs for <> 1
terminating with success status, rc 0
restic outputs for <> 1
restic outputs for <> 0
restic outputs for <> 1
terminating with success status, rc 0
restic outputs for <> 1

As you can see, only works fine on the first. After, Backup and delete old snapshots returns Error code 1.

Any help?

Thanks

Summary

This text will be hidden

2

When a restic command fails, then restic prints some information to stderr. The script currently either redirects errors to /dev/null or not at all (the restic backup command only redirects stderr). Please fix your script to capture the error messages. Then we can take a look at what goes wrong.

Btw, cron normally sends you a mail when a cronjob generated output. That is you might want to take a look where that went.

3

Ok… I modified the script:

#!/bin/sh

##Variables

now=$(date +%s)

start=$(date +%s)

##Cleaning Backups

find ~/Backup -name ‘.DS_Store’ -type f -delete > /dev/null 2>&1

##Cleaninig Restic Snapshots

/usr/local/bin/restic cache --cleanup

echo “restic outputs for <>” $? >> ~/System_Scripts/error.log

##Forget old snapshots

/usr/local/bin/restic --password-file ~/.pera -r rclone:gdrive:1 forget --keep-last 30 --prune

echo “restic outputs for <>” $? >> ~/System_Scripts/error.log

##Doing backups

#Backup Borg

export BORG_PASSPHRASE=$(cat ~/.melocoton)

/usr/local/bin/borg create ~/Backup_enc::$(date ‘+%A-%d-%B-%y-%T’) ~/Backup ~/1–show-rc --compression auto,lzma,6 2>> ~/System_Scripts/error.log

export BORG_PASSPHRASE=""

#Backup Restic

/usr/local/bin/restic --password-file ~/.pera -r rclone:gdrive:1 backup ~/Backup_enc/ >> ~/System_Scripts/error.log

echo “restic outputs for <>” $? >> ~/System_Scripts/error.log

#Backup Output

finish=$(date +%s)

elapse=$((finish - start))

printf “\n >>>>>>>>>Executed in $elapse secs at $(date). Done Backup”

printf “\n >>>>>>>>>Executed in $elapse secs at $(date). Done Backup” >> ~/System_Scripts/backup.log

Also… I modified the cron:

TZ=Europe/Madrid

*/15 * * * * /~System_Scripts/backup.sh >> ~/System_Scripts/error_cron.log

This is the output:

cat error_cron.log

no old cache dirs found

Executed in 24 secs at Sat Sep 4 15:45:25 CEST 2021. Done Backupno old cache dirs found

Executed in 7 secs at Sat Sep 4 16:00:07 CEST 2021. Done Backup

cat error.log

restic outputs for <> 0
restic outputs for <> 1
terminating with success status, rc 0
restic outputs for <> 1
restic outputs for <> 0
restic outputs for <> 1
terminating with success status, rc 0
restic outputs for <> 1

cat backup.log

Executed in 24 secs at Sat Sep 4 15:45:25 CEST 2021. Done Backup
Executed in 7 secs at Sat Sep 4 16:00:07 CEST 2021. Done Backup

So… The first thing I saw is prune doesn’t work when it has nothing to do. I guess this is the first error.

No idea about the backup issue…

Thanks

4

That still doesn’t capture stderr for most restic calls (only stdout). Also, please enclose code snippets in triple backticks ``` (on the line before and after) to make them readable:

snippet

forget --keep-last 30 --prune only runs prune when some snapshots were deleted. There’s no point in running prune otherwise.

5

ok thanks.

Uhm… I think my skills are not enough for doing it better…

Can you please suggest the modifications for it on Cron output?

6

What about the following?

*/15 * * * * ~/System_Scripts/backup.sh >> ~/System_Scripts/error_cron.log 2>&1
7

Great…Done.

This is what I have:

cat error_cron.log

no old cache dirs found
Fatal: unable to open repo at rclone:gdrive:1: exec: “rclone”: executable file not found in $PATH
Fatal: unable to open repo at rclone:gdrive:1: exec: “rclone”: executable file not found in $PATH

Executed in 8 secs at Sat Sep 4 23:10:08 CEST 2021. Done Backupno old cache dirs found
Fatal: unable to open repo at rclone:gdrive:1: exec: “rclone”: executable file not found in $PATH
Fatal: unable to open repo at rclone:gdrive:1: exec: “rclone”: executable file not found in $PATH

cat backup.log

Executed in 8 secs at Sat Sep 4 23:10:08 CEST 2021. Done Backup
Executed in 7 secs at Sat Sep 4 23:20:07 CEST 2021. Done Backup%

cat error.log

restic outputs for…cleaning… 0
restic outputs for…Forget old snapshots… 1
terminating with success status, rc 0
restic outputs for…Backup… 1
restic outputs for…cleaning… 0
restic outputs for…Forget old snapshots… 1
terminating with success status, rc 0
restic outputs for…Backup… 1

Seems like I have to add rclone to $PATH?. I don’t see why this works out of CRON and in CRON doesn’t…

Thanks

8

You’re right. And the reason is that the environment “inside” cron is different from the environment in your shell.

There’s plenty of articles on how to set or reference your desired environment or PATH inside a crontab, check e.g. this answer.

An alternative to setting the PATH variable manually is to instead start the process you want to run using e.g. bash, which will then make sure that the environment is more like you expect it to be, like in this answer.

9

Uhm… Thanks" I choose the 2nd option… and everything now works fine.

Thanks a lot!