Does anyone have any thoughts on finding differences between a given snapshot and the current files on disk?
What I have thought of so far:
mount and diff – but I’d prefer not to do that as I want something that works under termux on Android also)
restic dump as tar then pipe it to tar –diff, but this would be much slower than I would like, because the vast majority of files have not changed and ought to “pass” just on size and mtime if there were a tool that did this right.
So… does anyone know of a tool like that? Or a script? Or some other way to do this that I missed?
I think the output of restic ls <snapshot_id> --json should be enough to enable you to write some sort of a scripted solution, without having to do a full dump or mount of the snapshot.
There’s enough data contained in the json output to let you compare permissions, size, mtime, etc.
First, it depends on whether you want to diff on file- chunk/blob- or byte-level.
Moreover, I think there are different use-cases
if you want to check which files changed locally w.r.t to the latest backup/snapshot, (something like git status) the metadata in trees should be sufficient, i,e, a solution like @shd2h proposes.
if you want to also diff on chunk-level, you need to chunk and hash files and compare this with the data ids saved in the trees. With restic, this can be done by running dry-run backups or restores - but this needs carefully selected settings, is complicated and error-prone. Note that your depicted diff or dump solutions in fact are nothing else than dry-run restores but have the drawback that they need to download (maybe multiple times) all blobs from the repository which can be time-consuming or even expensive in the case of remote repositories.
if you want to diff on byte-level this gets very complicated unless you really use some restore-mechanism with the drawbacks depicted above.
if you want to check your local files for bit-rot, you need to do like above, but compare the chunks only for unchanged files and only report “unchanged” files with changed content.
Thanks. For the moment I settled on (a) ls –json piped to jq with appropriate selection on the restic side, andn the find command with an appropriate printf on the local disk side. Some minor massaging is required for the timestamps to look similar but that’s easily done.
The two outputs are then diff-able (and vimdiff`-able).
I also found a tool called gdu-diff (https://github.com/lilydjwg/gdu-diff) which takes two NCDU output files in JSON, compares them and produces an output JSON file which can be read back in GDU. So you get a nice interface to the diffs – the same interface that NCDU and GDU use. Blew my mind, and I can’t believe I never heard of it till now!
thank you. Yes, I only want a “git status” type of diff; easy enough, it turns out, (in hindsight!)
I tried to see what rustic does, but never got past the starting gate. Why does it require a TOML config file when I’ve already supplied the repository name and the password command through command line arguments or environment variables? The sample config file in the quickstart documentation only had repository and password anyway.
I’ll try it later, when I have some free time, but right now I’m good with the solutions I outlined in the other reply I made, especially gdu-diff
