Search Strategy when not knowing exact details

After reading “Backup Strategy - Help” the following scenario came to my mind:

  • Having a repo “which is NOT pruned and contains all historical data (just in case!)”
  • Many snapshots exist
  • Some day you miss some file
  • You don’t remember its exact name
  • You don’t remember when it was seen last, but are sure it has existed
  • You guess it must have been accidentally deleted
  • You have some ideas, where in your directory it might have been

What would be an efficient way to search the restic repo for that file?
Is there anything inbuilt in restic that might help with this?

1 Like

I don’t think there’s an efficient way to handle this but I’d probably write a bash script to restic diff over all snapshots and grep for deleted files.

What about restic find :wink:

$ restic find --help

The "find" command searches for files or directories in snapshots stored in the
repo.
It can also be used to search for restic blobs or trees for troubleshooting.

Usage:
  restic find [flags] PATTERN...

Examples:
restic find config.json
restic find --json "*.yml" "*.json"
restic find --json --blob 420f620f b46ebe8a ddd38656
restic find --show-pack-id --blob 420f620f
restic find --tree 577c2bc9 f81f2e22 a62827a9
restic find --pack 025c1d06

Flags:
      --blob            pattern is a blob-ID
  -h, --help            help for find
  -H, --host host       only consider snapshots for this host, when no snapshot ID is given
  -i, --ignore-case     ignore case for pattern
  -l, --long            use a long listing format showing size and mode
  -N, --newest string   newest modification date/time
  -O, --oldest string   oldest modification date/time
      --pack            pattern is a pack-ID
      --path path       only consider snapshots which include this (absolute) path, when no snapshot-ID is given
      --show-pack-id    display the pack-ID the blobs belong to (with --blob or --tree)
  -s, --snapshot id     snapshot id to search in (can be given multiple times)
      --tag taglist     only consider snapshots which include this taglist, when no snapshot-ID is given (default [])
      --tree            pattern is a tree-ID

I am working on systemd service/timer that does this:

backup.service - done

  • restic does the backup
  • rclone syncs the restic backup offsite
  • restic diffs the last two snapshots files, awk filters the files deleted from the snapshot which are then appended to a file. Command: ExecStart=/bin/bash -c 'exec /usr/bin/restic diff $$(exec /usr/bin/restic snapshots --json | exec /usr/bin/jq -r \'.[-2:][].id\') | exec /usr/bin/awk \'/^-/{for (i=2; i<NF; i++) printf $$i " "; print $$NF}\' >> $$RESTIC_DELETED_LOG' (stole part of this command from Script "restic diff" between the last two snapshots).

Updated the awk command to stop at the first separator (space character) in order to print the whole file, even if the file includes multiple spaces in its name.

backup-deleted-files-notify.service - wip

  • more manipulation will happen here as I need to filter the important directories
  • the consolidate, filtered list of deleted files will be emailed out daily at 20 o’clock

prune-check.service - done

  • run weekly
  • apply forget policy and prune backup
  • check the backup after pruning

Some References:


3 Likes

Thanks, this sounds like a good approach :slight_smile:

Cool, this might get you warned about unwanted deletions quite early. Thanks. :slight_smile:

1 Like