Any tips on diffing against current (on disk) files?

Hi all

Does anyone have any thoughts on finding differences between a given snapshot and the current files on disk?

What I have thought of so far:

  • mount and diff – but I’d prefer not to do that as I want something that works under termux on Android also)
  • restic dump as tar then pipe it to tar –diff, but this would be much slower than I would like, because the vast majority of files have not changed and ought to “pass” just on size and mtime if there were a tool that did this right.

So… does anyone know of a tool like that? Or a script? Or some other way to do this that I missed?

thanks!

I think the output of restic ls <snapshot_id> --json should be enough to enable you to write some sort of a scripted solution, without having to do a full dump or mount of the snapshot.

There’s enough data contained in the json output to let you compare permissions, size, mtime, etc.

Hi @sc2maha!

First, it depends on whether you want to diff on file- chunk/blob- or byte-level.

Moreover, I think there are different use-cases

  • if you want to check which files changed locally w.r.t to the latest backup/snapshot, (something like git status) the metadata in trees should be sufficient, i,e, a solution like @shd2h proposes.
  • if you want to also diff on chunk-level, you need to chunk and hash files and compare this with the data ids saved in the trees. With restic, this can be done by running dry-run backups or restores - but this needs carefully selected settings, is complicated and error-prone. Note that your depicted diff or dump solutions in fact are nothing else than dry-run restores but have the drawback that they need to download (maybe multiple times) all blobs from the repository which can be time-consuming or even expensive in the case of remote repositories.
  • if you want to diff on byte-level this gets very complicated unless you really use some restore-mechanism with the drawbacks depicted above.
  • if you want to check your local files for bit-rot, you need to do like above, but compare the chunks only for unchanged files and only report “unchanged” files with changed content.

A tool supporting all use-cases except byte-wise diffs is GitHub - rustic-rs/rustic: rustic - fast, encrypted, and deduplicated backups powered by Rust (with rustic diff) - which (disclaimer) I’m the maintainer of :slight_smile: