Does "restic snapshots" always list chronologically?

Getting Help
1

v 0.14.0, OS W10

Empirically this appears to be the case. But I haven’t found anything in the documentation which states/guarantees this.

The context is that I use a Python script and multiple subprocess.run() commands to carry out restic commands and then to analyse the results. At the moment I am taking the trouble to convert the timestamp string in the JSON output to a datetime.datetime (and then sort). But I suspect I don’t need to.

2

I don’t have an answer for you, but I believe there is an option to have Restic output JSON for you so you don’t need to convert it yourself.

3

I more than believe it does. But yeah, as @nstgc said, try

restic snapshots --json

and you should get nice data to work with.

EDIT:
sorry, I over read that you wrote you already use the JSON output.

4

Oops. I did too. I misread the opening post, thinking it said output timestamps were being converted to JSON, not that the JSON was being parsed. Sorry.

5

No worries! Easy mistake to make. Yes, using the --json flag and parsing the JSON ‘time’ key is straightforward enough.

6

It does, see restic/cmd_snapshots.go at master · restic/restic · GitHub

But note that the standard CLI output is mainly meant for humans and be prepared that it can be changed any time. To access data from a script, currently the JSON interface is IMO the preferred way. (I hope I manage to soon split rustic into a CLI and a library, but this is not directly restic-related)

7

Great thanks… um, yes, as previously discussed in this thread, I am already most definitely using the --json flag, not sure how a contrary impression was imagined.

8

phyrrhic might be interesting for you as you use python.

However, on a short look it seems that with pyrrhic you are restricted to a local backend and I think it’s not speed- or memory-usage- optimized. However, for just reading and processing snapshot files it should suffice, IMHO.

9

phyrrhic might be interesting for you as you use python.

Indeed sorting is just idiomatic python code using snapshot properties. Example code:

from operator import attrgetter
from pathlib import Path
from pprint import pp

from pyrrhic.repo.repository import Repository, get_masterkey

repopath = Path("restic_test_repositories/restic_test_repository")
masterkey = get_masterkey(repopath, "password")
repository = Repository(repopath, masterkey)

pp(sorted(repository.get_snapshot(), key=attrgetter("time")))

Results in:

[Snapshot(time=resticdatetime(2022, 7, 19, 21, 52, 28, 692251, tzinfo=datetime.timezone(datetime.timedelta(seconds=7200))), tree='a5dbcc77f63f5dd4f4c67c988aba4a19817aaa9d6c34a6021236a5d40ce653e1', paths=['/home/juergen/shared/python/pyrrhic/test'], hostname='shaun', username='juergen', id='dd62b535d10bd8f24440cc300a868d6bf2f472859f1218883b0a6faca364c10c', uid=1000, gid=1000, excludes=None, tags=None),
 Snapshot(time=resticdatetime(2022, 9, 25, 13, 17, 3, 310345, tzinfo=datetime.timezone(datetime.timedelta(seconds=7200))), tree='1eebeb922aa237d149209e6e9786655898dfcabaeeae1f79b8bfda686d3788fb', paths=['/usr/share/cracklib'], hostname='lemmy', username='juergen', id='fb56c7b68e95806244b0080bf8bb05d0132df9228390e4b9dc36e2ba6d3bf1a8', uid=1001, gid=1001, excludes=None, tags=None),
 Snapshot(time=resticdatetime(2022, 10, 17, 19, 24, 47, 151121, tzinfo=datetime.timezone(datetime.timedelta(seconds=7200))), tree='15b4341aa4e0dbc3414268fa2d15677f16dcdd6db7e03b8221aa57cfaaf9b2fe', paths=['/usr/share/cracklib'], hostname='shaun', username='juergen', id='7f9faf70a9889f54124f52e42f0d11d3e5eab185fe423cd2c4bb859ef0f71a8b', uid=1000, gid=1000, excludes=None, tags=None),
 Snapshot(time=resticdatetime(2022, 10, 21, 7, 29, 10, 262317, tzinfo=datetime.timezone(datetime.timedelta(seconds=7200))), tree='d29104863ce809c7f05dc3092176c5d3727b6941551812a2400a849fceea5866', paths=['/usr/share/cracklib'], hostname='shaun', username='juergen', id='d2bbc914bf89a24f16bcb70a4ec1d433856e0a8fef5b73c7dd4fa64a95f3977c', uid=1000, gid=1000, excludes=None, tags=['symlink'])]

However, on a short look it seems that with pyrrhic you are restricted to a local backend

Yes, I plan to add remote repositories in future versions (rclone).

I think it’s not speed- or memory-usage- optimized

Amazingly, some operations are even faster. The low-level libraries pyrrhic uses are all implemented in C. Listing snapshots:

time pyrrhic  snapshots
... 
real	0m0,925s
user	0m0,883s
sys	0m0,040s

vs

time restic  snapshots
...
86 snapshots

real	0m1,133s
user	0m0,788s
sys	0m0,040s
1 Like
10

Indeed that looks amazing. I actually don’t think that your goal is to outperform restic, but rather to provide a possibility to access the repo from python, right?

To be fair, listing snapshots is not something which needs heavy optimization and is mainly limited by accessing the snapshot files from disc (caching plays a big role here…). Out of interest, I did compare rustic with restic:

rustic ... snapshots
[...]
real	0m0,504s
user	0m0,471s
sys	0m0,058s

restic ... snapshots
[...]
514 snapshots

real	0m0,835s
user	0m0,619s
sys	0m0,061s

Then, I remembered that restic is writing and deleting a lock file which rustic (and pyrrhic, I think) is not writing. Using restic --no-lock instead shows much smaller differences between rustic and restic…

So, back to the topic: Thanks @juergenhoetzel for pyrrhic which helps a lot to access restic repositories with python!

1 Like