Recipe for seeing all files, in any snapshot

Getting Help
1

Looked here for a solution, couldn’t find one so I made one myself. Sharing for future Googlers.

I was having the problem of wanting to see all files, for any snapshot, for a given path. We want to use Restic to archive and manage/prune dated backup files, so we wanted to copy them to a dir, run restic backup, delete them, and trust in the restic pruning mechanism to keep the various versions apropriately long.

Only problem: doing that, any given snapshot only has one or two files (depending on when the backups and restic run), with no way to see what backups we actually have, other than the snapshot metadata. So we needed an ls variant that goes through all snapshots (which is of course horribly slow).

I tinkered the following together (needs jq):

for SNAPSHOT in $(restic snapshots --path /home/joerg/temp --json | jq -r '.[].id') ; do restic ls $SNAPSHOT '/' --json | jq -r '. | select ( .mtime != null ) | .path' ; done | sort -u

Example usage looks like this:

[root@braemar temp]# touch 1 && restic backup .
repository c755362d opened successfully, password is correct
using parent snapshot 6f628ba6

Files:           0 new,     1 changed,     0 unmodified
Dirs:            0 new,     0 changed,     0 unmodified
Added to the repo: 289 B

processed 1 files, 0 B in 0:29
snapshot db4984e6 saved
[root@braemar temp]# touch 2 && restic backup .
repository c755362d opened successfully, password is correct
using parent snapshot db4984e6

Files:           1 new,     0 changed,     1 unmodified
Dirs:            0 new,     0 changed,     0 unmodified
Added to the repo: 569 B

processed 2 files, 0 B in 0:32
snapshot bc71ad6a saved
[root@braemar temp]# rm -f 1 && restic backup .
repository c755362d opened successfully, password is correct
using parent snapshot bc71ad6a

Files:           0 new,     0 changed,     1 unmodified
Dirs:            0 new,     0 changed,     0 unmodified
Added to the repo: 292 B

processed 1 files, 0 B in 0:30
snapshot 004cad36 saved
[root@braemar temp]# rm -f 2 && touch 3 && restic backup .
repository c755362d opened successfully, password is correct
using parent snapshot 004cad36

Files:           1 new,     0 changed,     0 unmodified
Dirs:            0 new,     0 changed,     0 unmodified
Added to the repo: 292 B

processed 1 files, 0 B in 0:30
snapshot 6c4d2b5b saved
[root@braemar temp]# for SNAPSHOT in $(restic snapshots --path /home/joerg/temp --json | jq -r '.[].id') ; do restic ls $SNAPSHOT '/' --json | jq -r '. | select ( .mtime != null ) | .path' ; done | sort -u
/1
/2
/3
[root@braemar temp]#

Hope this helps someone.

1 Like