Restic Backup from macOS Apple APFS Snapshot of External Drive (without Time Machine)

I have found the thread Utilizing macOS local APFS snapshots for Restic backups to be very helpful. However, I had one problem with it. My volume is not part of a Time Machine backup. Thus, macOS does not create snapshots for it. It does not seem to be possible to use macOS tooling for creating APFS snapshots of such external drives. However, Carbon Copy Cloner ships a CLI that does the trick. (Unfortunately, it’s not free.)

I built a script that creates a snapshot; mounts it; starts a backup using the mounted snapshot; then removes the snapshot. It seems to work well. Maybe someone’s interested:

#!/bin/sh

set -euo pipefail
IFS=$'\n\t'

alias ccc="/Applications/Carbon\ Copy\ Cloner.app/Contents/MacOS/ccc"
alias restic="/opt/homebrew/bin/restic"

SNAPSHOT_VOLUME="OWC-AB-R1"
SNAPSHOT_MOUNT_POINT="/private/tmp/$SNAPSHOT_VOLUME-Snapshot"

RESTIC_HOST="Mac-mini.local"
RESTIC_INCLUDES_FILE="/Users/martin/.restic-includes"
RESTIC_EXCLUDES_FILE="/Users/martin/.restic-excludes"
RESTIC_CACHE_DIR="/Library/Caches/restic"
RESTIC_LIMIT_DOWNLOAD=62500 # 500 Mbps x 125 = KiB/s
RESTIC_LIMIT_UPLOAD=31250 # 250 Mbps x 125 = KiB/s

if [ ! -d "/Volumes/$SNAPSHOT_VOLUME" ]; then
	echo "⛔️ Aborting" >&2
	echo "Volume $SNAPSHOT_VOLUME is missing" >&2
	exit 1
fi

if [ -d "$SNAPSHOT_MOUNT_POINT" ]; then
	echo "⛔️ Aborting" >&2
	echo "Snapshot Mount Point $SNAPSHOT_MOUNT_POINT is already present" >&2
	exit 1
fi

echo "🗃️ Creating Permanent Snapshot for $SNAPSHOT_VOLUME"
if ! SNAPSHOT_LABEL="$(ccc --create "/Volumes/$SNAPSHOT_VOLUME" restic | grep -oE "com\.bombich\.ccc\.permanent\.[^ ]+$")"; then
	echo "⛔️ Aborting" >&2
	echo "Failed to create snapshot for $SNAPSHOT_VOLUME" >&2
	exit 1
fi
if [ -z "${SNAPSHOT_LABEL:-}" ]; then
	echo "⛔️ Aborting" >&2
	echo "Snapshot Label is empty — Unexpected Output from ccc" >&2
	exit 1
fi
echo "Snapshot Label: $SNAPSHOT_LABEL"

cleanup() {
	echo "🧹 Cleaning up"
	
	if mount | grep -q "$SNAPSHOT_MOUNT_POINT"; then
		echo "Unmount $SNAPSHOT_MOUNT_POINT"
		umount -f "$SNAPSHOT_MOUNT_POINT" || echo "🔔 Unmounting $SNAPSHOT_MOUNT_POINT failed" >&2
	fi
	
	if [ -d "$SNAPSHOT_MOUNT_POINT" ]; then
		echo "Remove Snapshot Mount Point $SNAPSHOT_MOUNT_POINT"
		rm -r "$SNAPSHOT_MOUNT_POINT" || echo "🔔 Removing $SNAPSHOT_MOUNT_POINT failed" >&2
	fi
	
	echo "Remove Permanent Snapshot of $SNAPSHOT_VOLUME"
	ccc --remove "/Volumes/$SNAPSHOT_VOLUME" "$SNAPSHOT_LABEL" || echo "🔔 Removing Permanent Snapshot of $SNAPSHOT_VOLUME failed" >&2
}

# Ensure cleanup runs on EXIT (no matter success or failure)
trap cleanup EXIT

echo "📂 Mounting Snapshot"
echo "Create Snapshot Mount Point $SNAPSHOT_MOUNT_POINT"
if ! mkdir -p "$SNAPSHOT_MOUNT_POINT"; then
	echo "⛔️ Aborting" >&2
	echo "Failed to create snapshot mount point" >&2
	exit 1
fi
# Mount with `mount_apfs` instead of `ccc --mount` to have a stable mount point path name
echo "Mount Snapshot $SNAPSHOT_LABEL to $SNAPSHOT_MOUNT_POINT"
if ! mount_apfs -s "$SNAPSHOT_LABEL" "/Volumes/$SNAPSHOT_VOLUME" "$SNAPSHOT_MOUNT_POINT"; then
	echo "⛔️ Aborting" >&2
	echo "Failed to mount snapshot $SNAPSHOT_LABEL" >&2
	exit 1
fi

echo "🗄️ Starting Backup Process"
echo "Source Restic Credentials"
. /Users/martin/.restic
echo "Start Restic Backup"
restic backup \
	--host "$RESTIC_HOST" \
	--files-from "$RESTIC_INCLUDES_FILE" \
	--exclude-file "$RESTIC_EXCLUDES_FILE" \
	--one-file-system \
	--cache-dir "$RESTIC_CACHE_DIR" \
	--cleanup-cache \
	--compression max \
	--limit-download "$RESTIC_LIMIT_DOWNLOAD" \
	--limit-upload "$RESTIC_LIMIT_UPLOAD" \
	-o s3.storage-class=INTELLIGENT_TIERING

echo "✅ Backup Completed Successfully"

Here’s the file contents of `/Users/martin/.restic`:

export RESTIC_PASSWORD_COMMAND='security find-generic-password -a resticGCS -s restic_pwd -w'
export AWS_ACCESS_KEY_ID=$(security find-generic-password -a resticGCS -s restic_aws_key -w)
export AWS_SECRET_ACCESS_KEY=$(security find-generic-password -a resticGCS -s restic_aws_secret -w)

export AWS_DEFAULT_REGION="eu-central-1"
export RESTIC_REPOSITORY="s3:https://s3.amazonaws.com/xxxxxxxxxxxxxx"

I have added these keys to the system keychain. The script runs via a global daemon as root:wheel.

I’m using LaunchControl. It ships with a tool called `fdautil`. This fixes full disk access (FDA) issues for me. (Also not free)

Disclaimer

Please use this at your own risk.

I’m not running this “in production” yet because I’m having issues with permissions when restoring. I don’t know if that’s a side effect of backup up from a mounted snapshot. I don’t believe, that’s the case though. I tried to backup files from another location and I ended up with the same issues.

1 Like

Nice idea with using CCC. Perfect for its users. Thanks for sharing.

But to use TM snapshots it is enough to include external disk in TM backup and exclude all its content. Snapshots are taken but no single file is stored in TM backup. However empty disk will be included. So not for OCD types:)

If anybody is interested fdautil works perfectly even with free version of LaunchControl. Other GUI based functionality is limited but what is left is still worth for macOS users playing with launchd.

1 Like

Awesome, thanks for sharing. I didn’t think about doing it this way. Maybe I’ll migrate my script to use `tmutil` to have a more “first party” solution.

If anybody is interested fdautil works perfectly even with free version of LaunchControl. Other GUI based functionality is limited but what is left is still worth for macOS users playing with launchd.

That’s also great news. However, LaunchControl is absolutely worth it imho. :slight_smile:

1 Like

Agree 100%. It is a real gem utility. I could not live without it. It is the heart of my restic backup orchestrator:)

CCC is awesome too, even though its usefulness diminished (at least for me) with Apple making boot volume copies functionality very limited.

2 Likes