Restic never finds a parent snapshot when the snapshot exists

I am running a nightly restic backup of my appdata directory. The snapshots
appear to be working. I can diff the snapshots and it only shows the changed
files. However, every snapshot says that it cannot find the parent snapshot.
When I run with -vv it finds files and marks them as “new” but it says “0 B
added, 0 B metadata”. Does anyone know why it wouldn’t find the parent
snapshot? I have pasted what I can tell is relevant below.

RESTIC_REPOSITORY  = "b2:<redacted>:/"
RESTIC_PASSWORD    = "<redacted>"
B2_ACCOUNT_ID      = "<redacted>"
B2_ACCOUNT_KEY     = "<redacted>"
RESTIC_CACHE_DIR   = "/var/lib/restic/cache"
AWS_ENDPOINTS      = "<redacted>"
AWS_DEFAULT_REGION = "<redacted>"
# restic snapshots
repository b09e5828 opened successfully, password is correct
ID        Time                 Host          Tags        Paths
----------------------------------------------------------------------------------------
95b3dd76  2021-09-01 10:43:09  7058dae0f56b              /var/lib/appdata
1f8a89fb  2021-09-02 05:00:14  4d3ae7c9b41b              /var/lib/appdata
6b768ceb  2021-09-03 05:00:04  2a9ababe2742              /var/lib/appdata
ac80dcf1  2021-09-03 08:34:38  1cfa87c70fcc              /var/lib/appdata
54b2477c  2021-09-03 08:54:00  efc839b46b8c              /var/lib/appdata
----------------------------------------------------------------------------------------
5 snapshots

As you can see the files are in the latest snapshot when I run restic ls latest

# restic ls latest | grep userPrefs
/var/lib/appdata/edifier/.java/.userPrefs
/var/lib/appdata/edifier/.java/.userPrefs/.user.lock.root
/var/lib/appdata/edifier/.java/.userPrefs/.userRootModFile.root
/var/lib/appdata/edifier/.java/.userPrefs/net
/var/lib/appdata/edifier/.java/.userPrefs/net/edifier
/var/lib/appdata/edifier/.java/.userPrefs/net/edifier/prefs.xml
/var/lib/appdata/edifier/.java/.userPrefs/net/prefs.xml

But when I do a backup it says it can’t find the parent and it shows those
files as new but unchanged.

# restic backup --iexclude-file=/var/lib/restic/excludes.txt /var/lib/appdata -vv
open repository
repository b09e5828 opened successfully, password is correct
lock repository
load index files
no parent snapshot found, will read all files
start scan on [/var/lib/appdata]
start backup on [/var/lib/appdata]
new       /var/lib/appdata/edifier/.edifier/tmp/, saved in 0.002s (0 B added, 0 B metadata)
new       /var/lib/appdata/edifier/.java/.userPrefs/.user.lock.root, saved in 0.002s (0 B added)
new       /var/lib/appdata/edifier/.java/.userPrefs/.userRootModFile.root, saved in 0.001s (0 B added)
...

Your host name changes with each backup run. restic only searches for parent snapshots with the same host name and path.

Try to set the hostname by setting --host or directly give the correct parent snapshot with --parent.

Ah. I did not know that. Is that documented somewhere? Why is the hostname considered?

restic supports backing up multiple hosts to one repository. If you backup the same path of different hosts, it is useful to use the last snapshot with that path and the same host as parent - instead of the last snapshot with only a matching path (this might be the last backup of a different host)

But you are right, in the case that no parent is found, a parent with same path but different hostname might be still a good guess - feel free to open a github issue to request this as a future feature!

1 Like

In the end this is not a big problem - the difference between finding a parent snapshot and not finding one is that in the latter case restic will have to rescan the files you back up to see if anything changed, whereas in the former case restic can instead just look at the metadata for the files to determine if they changed (assuming a trustworthy filesystem).

So, in the cases where this extra scanning is a problem because of a large dataset or similar, one can deal with it in other ways. The default is good as it is.

The help text for --parent in restic help backup mentions the default, yes. But it might be a good idea to explain the “parent” thing more explicitly in the documentation.

Thank you both for the explanation. I didn’t notice much of a performance issue, however there are thousands of files so I wasn’t sure what the impact would be later on. I set the hostname in docker and the problem went away.

I have a repo I’ve been running restic backup against nightly for three years by the way. It has saved me a number of times. Restic is one of my favorite pieces of software. Keep up the awesome work.

3 Likes

Mine too, totally :slight_smile: Couldn’t live without it!

2 Likes

Saved me too! Restic is a jewel!

I also found this useful when trying to move a whole folder in my directory tree, without re-backing up everything.

For example, if you do this:

cd ~/path1/mydata
restic -r /srv/restic-repo backup .
mv ~/path1/mydata ~/path2/mydata
cd ~/path2/mydata
restic -r /srv/restic-repo backup .

…then in the second backup command, restic considers all the moved files as being new.

But if you do the same, except that you supply the snapshot ID from the first backup as the parent for the second backup, then moved (but unchanged) files aren’t re-backed up, so you save on time / bandwidth / space.

I think this relies on the fact that in both backups here, the files have the same path relative to the working directory that the command is executed from.