Not finding a parent snapshot, after container running restic exits

I’ve been running a rather large backups with restic (105TB). However, restic never finds a parent snapshot and always reads entire repo, which makes it a very slow process.

It could be related to my container existing unexpectedly. Running restic in a synology device that runs restic using the resticker (GitHub - djmaze/resticker: Run automatic restic backups via a Docker container.).

After each snapshot, I can see snapshots being created:
repository 4ea1c842 opened successfully, password is correct

ID        Time                 Host                                      Tags        Paths
c594d7ad  2021-11-05 23:21:06  ws12-ubuntu                                           /storage/videostore
5983adad  2021-11-19 11:22:19  ws12-ubuntu                                           /storage/videostore
bf27f9c8  2022-01-07 11:11:32  videostore                                            /storage/videostore
a403c51f  2022-04-08 14:15:33  videostore-backup-first-day-of-the-month              /data
fe36ffd0  2022-04-27 11:35:16  videostore-backup-first-day-of-the-month              /data
5 snapshots

(The last two snapshots are created with the resticker)

However, I have not been able to see restic find the parent snapshot, and I am unsure why.
Any ideas why would that be?

Afaik paths and hostname was used for selecting parent snapshot, but I am not totally sure about it (& couldn’t find it on docs in a quick search).
If I understand correctly, hostname changes on every run since you’re inside a container. Could you try enforcing a stable hostname with --hostname flag while giving the backup command?

With which parameters is restic run exactly? Juding from the snapshots output fe36ffd0 might have a403c51f as parent.

Sorry for the completely wrong report.

When I started the container for the third time, I read the beginning of the operation carefully, and restic did find the parent snapshot. I was mistaken because the second backup snapshot seemed to be as slow as the first one.

The operation still takes about 12-13 days (even if no new files added). Sometimes files that are already uploaded seem to just take minutes, even though I would imagine they would be cached.

I run restic like this:
restic --repo=b2: backup --exclude=/data/builds /data

Anything I could do to speed backup up? How could I make sure that the parent snapshot and caching works as intended?

How much data does restic add to the repository for later backup runs? Is the repository cache shared across container runs? Or does restic have to download it each time? How large does the cache folder become?

When restic detects the proper parent snapshot, then it will be able to skip files that have not changed as long as the file metadata has not changed. Which filesystem is used for /data? 12-13 days is approximately what I get when dividing 100TB/100MB/s . So it looks like restic has to read everything again? How much CPU does restic use?

The restic doesn’t add much data between the runs. It does look like it is reading all files again (hence my initial suspicion of not finding the parent snapshot).

Restic practically doesn’t use any CPU (5% or so on the new run).

The /data is mapped into the container from outside of the container. The filesystem of /data inside and outside of the container is cifs.

Cache folder for some reason is very small.
This is from inside the container now:.

bash-5.1# ls -la
total 4
drwx------    1 root     root           152 Mar 14 08:52 .
drwx------    1 root     root            12 Mar 14 08:52 ..
drwx------    1 root     root            50 May 20 11:34 4ea1c84219c682f4ceda40c0cf41a24becc73548331cd3cea2341d44aa6908b1
-rw-r--r--    1 root     root            44 Mar 14 08:52 CACHEDIR.TAG
bash-5.1# pwd

And this is what mount shows for the /data inside the container:
\134\13410.1.10.8\134videostore on /data type cifs (ro,relatime,vers=3.02(syno),blocksend,sec=ntlmssp,cache=strict,username=XXXXXXX,domain=WORKGROUP,uid=1026,forceuid,gid=100,forcegid,addr=,file_mode=0777,dir_mode=0777,iocharset=utf8,nocase,nounix,mapposix,rsize=1048576,wsize=1048576,actimeo=1)

I am on restic version
restic 0.12.1 compiled with go1.16.6 on linux/amd64

My guess right now would be that CIFS provides unstable inodes. Does adding --ignore-inode help?


OK, thanks for the suggestion. I will give it a go when the current job finishes and report back.
Cache is 39.2Gb of size btw.

Even with the 13 day snapshotting of the “oversize” repository I am finding restic useful, so thank you.

Yes, the --ignore-inode helped my case a lot. The backup time for my huge repository went from 13 days to under 1 hour when I used that option.

So now, the backup times are very reasonable IMO even for the big repo. Thanks again for your help.