Possible bug in stats?

Getting Help
1

Environment: 2 identical machines
Both running restic 0.15.2 compiled with go1.20.3 on linux/amd64
Both backing up to the same remote repo

I observe that on both machines, running ‘restic stats lastest --host A’ reports a size of 24.604 MiB.
The expected output should be something like 6 GiB.

Why do I have that expectation? Because the latest snapshot contains a file that is ~4 GiB which gets written just before the snapshot is created. This file (a tar file) seems to be correctly backed up: I restored it and checked md5sum: It’s the same for both the original and the restored file.

Things I noted

  • original file is read from ext4, restored file was saved to zfs
  • stat shows the same size for both files
  • md5sum is the same for both files
  • du on both files reports the same “apparent size”, but a different device usage (3,8GiB vs 1,8GiB - maybe because of the different FS?)
  • Nevermind the different size reported by “du”, restic should still not tell me the snapshot is “24 MiB”.

Do I have a wrong expectation, perhaps regarding the output of the stats command?

2

We’re probably experiencing the same thing: Restic shows huge amount of data being backed up while there is none - #2 by aerbil313

Edit: Mine is apparently the case of virtual filesystems etc. being included resulting in more size showing up than it should. Yours I don’t know.

3

The default mode of stats should output the file size after a restore operation.

Can you provide the full output of restic stats latest --host A ? Please also check the output of restic ls --host A -l latest. How did you specify the snapshot for the restore command, also as latest --host A?

4 
repository 0214ba19 opened (version 2, compression level auto)
scanning...
Stats in restore-size mode:
     Snapshots processed:  1
        Total File Count:  1700
              Total Size:  24.604 MiB

snapshot c9da2142 of [... /media/bindmounts/usbhdd/backup] filtered by [] at 2023-07-18 01:36:32.583057236 +0200 CEST):
...
-rw-r--r--     0     0 3577497600 2023-07-18 01:36:08 /media/bindmounts/usbhdd/backup/myfile-2023_07_18-01_30_00.tar
...

Yes, I did just that.

5

@MichaelEischer: Anything I can do to get to the bottom of this? I can provide all kinds of logs or debug output, if necessary.

6

Is the file count reported by restic stats latest smaller than of restic ls -l latest | wc -l by one?

$ restic stats latest
scanning...
Stats in restore-size mode:
     Snapshots processed:  1
        Total File Count:  3574
              Total Size:  135.471 MiB
$ restic ls -l latest | wc -l
3575

There’s currently no debug output in the stats command to would explain what’s going on.

7

I confirm: File count by stats latest is 1 less than by ls -l latest.
I’ll start backing up into a new repo to be sure everything’s fine.

8

Update: It just happened again with the same 2 hosts backing up to a new repo.
restic diff (latest, size is off vs. the previous one, where the size is normal) shows the files correctly (especially the large tar file being ‘replaced’ [old file -, new file +]). Size seems correct, too:

Added:   1.089 GiB
Removed: 1.031 GiB

Still, restic stats latest shows the snapshot size to be 28.435 MiB.