Why does restic upload so much metadata for small, unchanged folders, e.g., `(0 B added, 659.680 KiB metadata)`?

I use restic 0.12.1 on macOS 12.1 (2021 MacBook Pro). Occasionally, when doing restic backup $HOME, there will be hundreds of directories which are “modified” and restic uploads large amounts of metadata. Here’s an example:

modified  /Users/quinn/Archive/Fonts/Advertisers GothicLight/, saved in 93.934s (0 B added, 129.000 KiB metadata)
modified  /Users/quinn/Archive/Fonts/Arquitectura/, saved in 93.932s (0 B added, 61.854 KiB metadata)
modified  /Users/quinn/Archive/Fonts/Avant Garde (Adobe)/, saved in 93.932s (0 B added, 659.680 KiB metadata)

These directories are never touched, and nothing ever changes in them. In the example above, I haven’t opened these Fonts/* folders in many years. The first two directories contain two files each, and the third one contains only 15 files. Notice the third one is uploading 659.680 KiB metadata. That is weird because the total size of that directory and all its contents is only 520 KiB. :man_facepalming:

The “modified” metadata uploads don’t happen every time I run restic, but it does happen more often than I wish it did. Perhaps it happens when I back up after restarting my computer?

Does anyone have any idea what’s going on here?

Your filesystem/system could keep track of the last access date, which would be the time/date a file has been opened for reading or writing.

In general keeping track of the “adate” usually happens at one of three levels:

  • Always: each and every access to a file will be tracked and change the metadata, so restic might actually cause changing the metadata by itself, though then it would occur on each update
  • Once a day: only the first occurrence each day will be tracked. This is often the default on moder Linux distributions, not sure about Mac.
  • Off: adate won’t be changed ever after file creation.

There might be other metadata changing, though I have not yet the knowledge to know what is tracked at all.

PS: “always” is about never used today, as it has noticeable performance implications

Maybe related to Backing up (macOS) Photos Library, especially:

I’ve had similar issue: Restic seems to consider some files changed even though the files have just been laying on the disk. @NobbZ’s hypothesis about adate sounds reasonable.

Is there a way to tell restic to ignore adate of files? It seems it should only track mdate or cdate, since those indicate file change.

Does this in any way help? Some documentation for restic’s change detection.

The adate is ignored by default, unless you pass the --with-atime option to the backup command.