Directories change with every backup

Hi, I’m new to restic and when I do backups I always get that two directories are changed.

I’m trying to backup files that are in my iCloud folder on macOS 14.5 Sonoma with restic 0.16.4 compiled with go1.21.6 on darwin/arm64. File system is APFS

This is the command I’m using

/opt/homebrew/bin/restic -r sftp://user@host.tld:23//home/m-mac-01/icloud_main --verbose=2 --password-file /path/to/passwordfile backup "/Users/toor/Library/Mobile Documents/com~apple~CloudDocs/_main"

With the verbose output I see that the directories /Users/toor and /Users are marked as modified.

[... more unchanged output ... ]
unchanged /Users/toor/Library/Mobile Documents/com~apple~CloudDocs/_main/screenshots/
unchanged /Users/toor/Library/Mobile Documents/com~apple~CloudDocs/_main/
unchanged /Users/toor/Library/Mobile Documents/com~apple~CloudDocs/
unchanged /Users/toor/Library/Mobile Documents/
unchanged /Users/toor/Library/
modified  /Users/toor/, saved in 0.471s (0 B added, 0 B stored, 0 B metadata)
modified  /Users/, saved in 0.472s (0 B added, 0 B stored, 300 B metadata)

Files:           0 new,     0 changed, 12726 unmodified
Dirs:            0 new,     2 changed,  2533 unmodified
Data Blobs:      0 new
Tree Blobs:      2 new
Added to the repository: 745 B (600 B stored)

processed 12726 files, 6.855 GiB in 0:01
snapshot de99faf7 saved

I tried with --ignore-ctime and --ignore-inode which also always shows the 2 modified directories. It happens, even if I immediately execute the backup after the last one is finished.

stat for directories

Am I doing something wrong or is this a bug / MacOS issue?

In the example above, your /Users/ dir had 300 bytes of metadata changed since the last snapshot, hence the message.

Your /Users/ folder on MacOS (and most other OSs) which houses your homedir is subject to constant accesses; every few seconds is the norm. The mere fact you have run a restic command as a user in that folder will cause changes to files / metadata in that folder.

So, to be expected.

Is this causing a particular issue with your backups?

thanks for your reply. I’d like to set up a cron job that runs every 2 hours and this would generate a snapshot regardless of changes. I’d like to only have a snapshot when there are changes, even if this isn’t necessarily relevant on a technical level.

I was hoping there was an option to ignore folder changes down the path that aren’t relevant for the backup.

Something about those folders seems to change. Please provide the output of the following command for two snapshots, replace $snapshotId with the actual ID of the snapshots:
restic cat tree $snapshotId:/

1 Like

Hi Michael, this is the output:

restic cat tree $snapshotId:/

repository cfbfe9b9 opened (version 2, compression level auto)
[0:00] 100.00%  8 / 8 index files loaded
{"nodes":[{"name":"Users","type":"dir","mode":2147484141,"mtime":"2024-05-14T22:26:51.753291531+02:00","atime":"2024-05-14T22:26:51.753291531+02:00","ctime":"2024-05-14T22:26:51.753291531+02:00","uid":0,"gid":80,"user":"root","group":"admin","inode":15812,"device_id":16777229,"content":null,"subtree":"a8d817789c6efa27bd2dedb18abf019b4986041b83acc6b5f900efd419599f96"}]}
repository cfbfe9b9 opened (version 2, compression level auto)
[0:00] 100.00%  8 / 8 index files loaded
{"nodes":[{"name":"Users","type":"dir","mode":2147484141,"mtime":"2024-05-14T22:26:51.753291531+02:00","atime":"2024-05-14T22:26:51.753291531+02:00","ctime":"2024-05-14T22:26:51.753291531+02:00","uid":0,"gid":80,"user":"root","group":"admin","inode":15812,"device_id":16777229,"content":null,"subtree":"147e4eb0e44e8d374fbf4005b1625e59650d1e01c4245e56e383cd497024549a"}]}

Additional info: Just to outrule that it’s caused by the data of my real repository I did another test with one file that is also in /Users where I see the same behavior

Edit: Video has problems loading for me here, this is the link

Can you repeat that for restic cat tree $snapshotId:/Users? That should show some difference between two subsequent snapshots.

sure

cat tree 0867de27:/Users

repository cfbfe9b9 opened (version 2, compression level auto)
[0:00] 100.00%  15 / 15 index files loaded
{"nodes":[{"name":"toor","type":"dir","mode":2147484141,"mtime":"2024-06-02T21:38:19.153545182+02:00","atime":"2024-06-02T21:38:19.153545182+02:00","ctime":"2024-06-02T21:38:19.153545182+02:00","uid":501,"gid":20,"user":"toor","group":"staff","inode":25452,"device_id":16777229,"content":null,"subtree":"c4691eb50e4b053793b39b64a7b56401092dd0d115f390e92099f0aa5d4405b5"}]}

cat tree 6adcf297:/Users

repository cfbfe9b9 opened (version 2, compression level auto)
[0:00] 100.00%  15 / 15 index files loaded
{"nodes":[{"name":"toor","type":"dir","mode":2147484141,"mtime":"2024-06-02T21:38:24.562933858+02:00","atime":"2024-06-02T21:38:24.562933858+02:00","ctime":"2024-06-02T21:38:24.562933858+02:00","uid":501,"gid":20,"user":"toor","group":"staff","inode":25452,"device_id":16777229,"content":null,"subtree":"c4691eb50e4b053793b39b64a7b56401092dd0d115f390e92099f0aa5d4405b5"}]}

Hmm, the atime, mtime and ctime are different between both snapshots, with the additional oddity that all three times are identical. Seems like some macOS magic is going on.

If I’m not mistaken, then actual home partition is mounted at /System/Volumes/Data/home do you also see that weird timestamp behavior when backing up that path?

ok, thank you for checking. What a bummer that macOS is doing weird things. The /home directory isn’t usually writable

Maybe I misunderstood and this screenshot might help

My home directory /Users/toor has the same issue with backup (when I try to backup /Users/toor/temptest)

When I try to backup something from my external drive I it doesn’t happen

restic -r sftp://user@host.tld:23//extpath --verbose=2 --password-file /passkeyfile
backup "/Volumes/EXT1/testing"
open repository
repository cfbfe9b9 opened (version 2, compression level auto)
lock repository
using parent snapshot c6c59a91
load index files
[0:00] 93.75%  15 / 16 index files loaded
start scan on [/Volumes/EXT1/testing]
start backup on [/Volumes/EXT1/testing]
scan finished in 0.695s: 1 files, 8 B
unchanged /Volumes/EXT1/testing/backuptest.txt
unchanged /Volumes/EXT1/testing/
unchanged /Volumes/EXT1/
unchanged /Volumes/

Files:           0 new,     0 changed,     1 unmodified
Dirs:            0 new,     0 changed,     3 unmodified
Data Blobs:      0 new
Tree Blobs:      0 new
Added to the repository: 0 B   (0 B   stored)

processed 1 files, 8 B in 0:00
snapshot 85c467d6 saved

Sorry, the path I meant was /System/Volumes/Data/Users. AFAIK the folders in /Users are actually stored on /System/Volumes/Data/Users as the root partition is not writable on macOS.