Backfilling snapshots from non-restic archives

17

I don’t have much time right now (the days before Christmas are very busy here).

The path recorded for a snapshot is independent of how the structure in a snapshot looks like. The path is stored for several reasons:

  1. Try to find a previous snapshot on subsequent backups (what you can manually set with --parent)
  2. Group snapshots and treat different paths as different groups for the forget operation
  3. Display the origin of the data in a snapshot to users (e.g. with snapshots)

What you’re trying to solve (as far as I understood it) is mostly 2.

The problem is, even if you pass the right parent snapshot in manually, restic will likely detect the files as new because they are new (you’ve recently extracted them from a previous backup), so you don’t gain so much from the “incremental” mode. You can check what restic prints with -v -v or by inspecting the number of “modified” vs “new” files restic prints at the end of a backup run. There’s a PR (#2047) which will add the --ignore-inode option to the backup command which may help in this situation. If it doubt, restic will always opt to re-reading data, because that’s the safe thing to do.

I think that in the long run, using the paths displayed to users for 1 and 2 is not sufficient, and we need to come up with something different.

Now you may ask yourself, “how can I influence the structure within the snapshot?” In earlier versions, restic would only use the last path component from the path passed to restic backup as the top-level path component. For example:

$ restic backup /home/user /mnt/srv/other/bar

would lead to /user and /bar being the top-level path element in the snapshat. That led to all sorts of nasty problems (see #549 for a collection).

We changed this behavior in restic 0.9.0, so that it behaves like users expect it to behave, modeled after what tar does:

  • For absolute paths, restic recreates the same structure within the snapshot: restic backup /home/user/foo will build the structure /home/user/foo in the snapshot
  • For relative paths, restic will use the paths as they are passed to it: cd /home/; restic backup user/foo will build the structure /user/foo in the snapshot

Note that in boch cases the path recorded in the snapshot will be /home/user/foo. When restic is run again with this (absulute) path, it doesn’t know the structure within the snapshot. If the previous run used the same (absolute) path it’ll work and restic will be able to skip unmodified files. If the previous run used the relative path (second case above), the structure in the snapshot is different and restic will re-read all files. That’s why I think that we need to come up with something else to detect unmodified files eventually.

In conclusion, you can influence the structure within a snapshot by changing the current directory and passing in relative paths to restic backup.