Can restic backup macOS Photos library from launchd?

Getting Help
1

When I run restic from a commandline with sudo it is actually able to backup my Photos library. But this is probably because I have given Terminal full disk access.

I am actually starting restic from a script of my own. But it works fine.

I also have created a launchd schedule. But when it runs from there, I get these permissions errors, like:

scan: open /Users/gerben/Pictures/Photos Library.photoslibrary: operation not permitted

The launchd plist contains:

    <key>ProgramArguments</key>
    <array>
        <string>/opt/local/etc/restic/rna-run-restic.sh</string>

That script, which is multi-system, so can run on macOS and on Linux with docker, starts with

#!/bin/bash
# R&A restic interface script

# Make shure we're running bash >4
if [ -n "$BASH_VERSION" ]; then
    # Extract the major version number (first number) of BASH_VERSION
    major_version="${BASH_VERSION%%.*}"

    # Check if the major version is less than 5
    if [ "$major_version" -lt 5 ]; then
        # Re-execute the script with /opt/local/bin/bash
        exec "/opt/local/bin/bash" "$0" "$@"
    fi
else
    # Re-execute the script with /opt/local/bin/bash
    exec "/opt/local/bin/bash" "$0" "$@"
fi

And at some point on macOS, this launches /opt/local/bin/restic (installed by MacPorts) with the correct arguments, e.g. something like:

/opt/local/bin/restic --verbose -p -passwd.txt -r rest:https://fqdn:port/ backup --host foo --tag pictures --files-from /opt/local/etc/restic/etc/foo-includes-pictures.txt

But while from Terminal, I can run that backup, I cannot from launchd. How do I fix that?

2

OK, I seem to have found the ‘solution’: for this to work you must give bash full disk access. Don’t like that, but I do not know what a better working solution is.

1 Like
3

We have a bunch of Topics regarding FDA and macOS. Here is one where I figured out to not give entire Bash FDA – which you shouldn’t for obvious reasons and you already know that which is also good.

HTH

4

It seems GitHub - sveinbjornt/Platypus: Create native Mac applications from command line scripts. is a better solution

5

If you would’ve read the thread and my final post, you would’ve seen that I did in fact use that exact solution and additionally gave a step by step manual on how to build a working setup :wink:

6

You are right. I did not read that properly. Apologies.

7

No worries :slight_smile: I was just making sure you don’t spend more time on troubleshooting when we have threads that solve those issues.

The paths in my backup all start with /tmp/restic/backup where I have mounted the APFS snapshot which begins at / so all folders are accessible for me at the path /tmp/restic/backup as this is my “new” /.

My backup command that I use in my launchd entry looks like this:

restic backup \
  --compression max \
  --cleanup-cache \
  --exclude-caches \
  --no-scan \
  --quiet \
  --host Moes-MacBook-Air \
  "${backupdir}/Users/moritzdietz" \
  --files-from="${tempdir}/moritz-include.restic" \
  --exclude-file="${tempdir}/sys-exclude.restic" \
  --exclude-file="${tempdir}/moritz-exclude.restic" \
  --option s3.connections=10 \
  --read-concurrency 25

I highly recommend setting the host of the backup manually – too often I ended up with different hosts in the repo which makes it really difficult when it comes to cleaning up snapshots etc.

To your question I do backup my user home + additional files but also exclude files.
I usually never restore in place so I never ran into an issue with this.
If I wanted to have my backup look like I made it from /Users/moritzdietz/ instead there are several hoops to jump through which I don’t care about so I don’t do that.

But as mentioned in the other threads (both mine and the roadmap for the upcoming releases + the linked GH issues) this should mostly work.

Let us know if get it working in the end or need more help.

8

I am going to keep away from snapshots as long as that means I am getting the snapshot mount point in the paths in my macOS backup.

My own script is a wrapper around restic that works both on a Linux/docker setup as a macOS with MacPorts setup. I’ll probably report on it here later when I’m fully satisfied, but I can for instance let it install scheduled backups e.g.: rna-run-restic.sh --myhostid foo --destination b2 --tag foobar schedule Hour 4 Minute 23 installs:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>nl.rna.restic.calendar.foo.foobar.b2</string>
    <key>UserName</key>
    <string>root</string>
    <key>GroupName</key>
    <string>admin</string>
    <key>StandardOutPath</key>
    <string>/var/log/rna-restic/output.log</string>
    <key>StandardErrorPath</key>
    <string>/var/log/rna-restic/error.log</string>
    <key>ProgramArguments</key>
    <array>
        <string>/opt/local/etc/restic/rna-run-restic.sh</string>
        <string>--myhostid</string>
        <string>foo</string>
        <string>--tag</string>
        <string>foobar</string>
        <string>backup</string>
        <string>b2</string>
    </array>
    <key>StartCalendarInterval</key>
    <dict>
		<key>Minute</key><integer>4</integer>
		<key>Hour</key><integer>23</integer>
    </dict>
</dict>
</plist>

All secrets and such (e.g. AWS_ACCESS_KEY_ID or RESTIC_REST_USERNAME) in root-only readable files foo.env (so no passwords and such on command lines), includes and excludes for backups in files like foo-includes-foobar.txt etc. The thing I have to fix yet is to be able to run this from launchd without having to give bash full disk access and maybe a somewhat cleaner arg style (a bit of a mix now, but as it is already running on 8 systems of 5 people a cleanup that I might skip). Support for B2, restic’s own rest server, and local repositories. (Un)Schedules either backups or forget --prune. A lot of ‘sensible’ defaults (like a --host value when non is given, even if setting that explicitly is indeed better and is done by default when a schedule is created for instance).

Quite happy, especially with how efficient and fast restic is.

And the script makes it very easy for less technical users to add their own backups to their own repositories

(Until two weeks ago, I used to run my own MinIO server and have them use Duplicati to backup to it, I used CrashPlan to backup my server. Now they (and I) use my script around restic and backup to B2 (and I locally as well)).

(The plist shows a different/older way of giving the destination than with my current --destination flag, but that got in the way. Cleanup means I should remove that backwards-compatible behaviour, but that means everybody has to reschedule)

1 Like