Snapshot bigger than filesystem

Getting Help
1

Hi, I’m trying to backup my old macbook to a restic repository hosted on OneDrive via an encrypted rclone remote. I ran the following:

export RESTIC_REPOSITORY='rclone:onedrive-crypt:/restic'
export RESTIC_PASSWORD_COMMAND='/Users/J/restic/get-restic-password'
export RESTIC_COMPRESSION='auto'
export RCLONE_CONFIG='/Users/J/.config/rclone/rclone.conf'

restic unlock
restic backup / --exclude-file=/Users/J/restic/excludes.txt

This is the output showing the current progress (it’s also taking a really long time, but maybe that’s expected?):

[6:49:21] 11.19% 704196 files 53.986 GiB, total 4964281 files 482.561 GiB, 3 errors

(the 3 errors were related to some system files it couldn’t open that I don’t think are important for my purposes)

What I don’t understand is that this laptop only has a 256 gb ssd, so I don’t know where it’s finding 482gb of files to back up. Possibilities I’ve considered:

  • I was having a similar issue on my new laptop, backing up to this same repo, where the backups kept getting bigger, which I think was because I had the cloud drive where the repo is stored mounted on the filesystem, forgot to exclude it, and it caused a feedback loop trying to backup the old backups. I don’t think that’s the issue here because the drive shouldn’t be mounted anywhere on this filesystem.
  • Restic could be recursing into some other cloud storage. But I think the only other thing that’s connected is an iCloud drive with ~1gb of photos on it
  • I copied the exclude file I used from this github page which is intended for use with Borg. I’m not sure if Restic uses exactly the same format, so it might not be excluding files properly, but even if it’s failing to exclude anything at all the size of the backup shouldn’t exceed the size of the filesystem
  • The APFS filesystem has some built-in compression, so maybe some files appear bigger as Restic is reading them than the amount of physical space they take up on the drive because of this? My understanding of how any of this works is really limited but in any case almost twice as much space still seems like a lot to me
  • I have Windows installed on a Bootcamp partition on the laptop that I forgot to add to the list of stuff to exclude. Maybe this could be causing an issue but I’m not sure how

Output of “restic version”:

restic 0.17.3 compiled with go1.24.0 on darwin/amd64

I appreciate any help anyone is able to provide with this issue

1 Like
2

All what you mentioned is possible - nobody can guess without looking into details.

As a bare minimum exclude /Volumes directory where all Data volume is mounted and then transposed into root filesystem using firmlinks. Otherwise you backup things multiple times - it wont take space (thx to deduplication) but is very messy.

IMO there is no point in backing all macOS root file system as it is impossible to restore working macOS from such backup anyway. So not sure what is the value of storing this data.

Home folder only (/Users) is what is enough ( - caches, logs and similar exclusions.) in most cases. If you search this forum you will find other threads discussing macOS backup exclusions in more details.

2 Likes
3

One small idea: Might be the “time machine” thingy. Not sure how old the macbook is, but some app might be utilizing any kind of filesystem snapshots, it might show up as more total data. It is same with btrfs/snapper on Linux side.
Won’t be too much of a problem to restic since it will deduplicate as @kapitainsky said, but will mess up the metrics like this.

4

Very unlikely that it has anything to do with Time Machine. But what is almost certain is that it is very unwise strategy to start backing up “everything” without understanding what “everything” is.

As I said. On macOS - backup home folder only with carefully excluded all garbage. Other directories in root are almost transient and their backup has no use on modern macOS. Unlike Linux it is impossible to restore system itself. It has to be installed from source and cryptographically validated. Which even if it sounds difficult it super easy as installation can be bootstrapped from Internet. It is not full blown appliance yet but not far from it.

Only user data is something what should be protected using backups. Going beyond requires very deep knowledge of macOS internals.

Here you are macOS Sequoia volumes layout on Intel (as this is what OP is using):

image
image1347×904 113 KB

and directories mapping:

image
image1920×1294 129 KB

Source: Boot volume layout and structure in macOS Sequoia – The Eclectic Light Company

Only elements in blue are writeable and worth backing up. You can also see that parts of users data is accessible via root twice (/Volumes and /User).

5

Alright. Without having thought about it very much, I had figured I would back up as much of the system as I could for completeness/out of caution in case there was e.g. some configuration in /etc or something I might realize I wanted to reference at some point. But what you’re saying makes sense (and I’m realizing from those diagrams that I basically don’t understand the macOS file hierarchy at all) so I made a new backup of just /Users and it’s a much more reasonable size now.

Thank you for the info!

2 Likes