Lots of modified directories when actually not modified

Getting Help
1

Hi. I am using Restic 15.1 on Linux and at times restic thinks that most all directories are modified for no reason I can see.

Anyone know why this happens?

Thanks in advance for any suggestions.

2

This is usually caused by changed inodes (e.g., the inodes returned by the filesystem are not stable) or a changed device id (e.g., from different zfs/btrfs snapshots).

But we’ll need more information to anything definite. Which filesystem are you backing up? Please provide an example of the statistics printed by the backup command.

3

Thanks for your quick response.

OK, so hear is a normal backup.

Files: 219 new, 1293 changed, 4074566 unmodified
Dirs: 1 new, 227 changed, 376500 unmodified
Data Blobs: 847 new
Tree Blobs: 185 new
Added to the repository: 886.816 MiB (886.885 MiB stored)

Now, here is one where a lot of directories that were not changed are
marked modified.

Files: 22276 new, 48008 changed, 4010131 unmodified
Dirs: 12 new, 87411 changed, 289958 unmodified
Data Blobs: 51321 new
Tree Blobs: 84797 new
Added to the repository: 12.317 GiB (12.326 GiB stored)

I am using zfs for my file systems and backing up to the cloud. Most
of the directories in the second example are not actually changed as
far as I can see.

4

My guess would be that some metadata of the directories has changed, or that there was a change in their subfolders. Judging from the number of change directories my guess would be that they are clustered in some place. If you’re using backup -vv then the output should tell you which directories are the problem.

5

Yes, I am using backup -vv but why should so many directories be
marked as modified? Would zfs have anything to do with it? This
happens every so often witout any reason I can figure out. I thought
it might be a bug in restic.

6

Please run stat /path/to/some/changed/directory and compare it to the output of that command after there’s another spike in changed directories.

7

OK, I will try that – maybe zfs changes something.

Thanks.

8

OK, so it happened again, – I did a stat on one of the so-called
modified directories and the only difference I see is that when I did
it the first time it said Device: 0,60 Inode: 7 Links: 2
and now it says
Device: 0,58 Inode: 7 Links: 2

so, the only thing I can think of is that I did reboot between the
time I did the first stat and the second. I wonder is there a way to
tell restic to ignore the device – I am not sure why they should be
different, I am using zfs if that might be a problem.

Thanks.

9

That looks a lot like https://github.com/restic/restic/issues/3041 .

10

Thanks, interesting. One thing, I am not backing up using a snapshot,
so I am not sure what happens here – I wonder if zfs changes the
device id, and if so when does it do it?

11

I am using 15.2 at this point. What is the patch you had?

12

That could happen after a reboot depending on how the disks are initialized.

There’s no patch merged for the problem so far.

13

But my problems happen without a reboot, so I wonder if its exactly
the same – does sound related.

14

Is the disk maybe remounted while the system is running, e.g. using an automounter?

15

No, and I don’t know what triggers this, so I was hoping that the
device id would not be stored. I do have snapshots, but they are not
usually mounted at all.