Slow snapshot listing on s3

Hi again :wave:

On repos with a lot of snapshots, a simple command like restic snapshots --latest 1 -H server1 takes considerable amount of time. It has to look up all the snapshots (one-by-one? at least on s3 backend) to find the snapshot I am looking for.

Am I doing something obviously wrong, or is this an area of improvement? It could request more object at once, or we could have an index of snapshots to find it faster (though I am not sure how to be backwards-compatible with this one).

That is essentially issue [architecture] Performance of `restic snapshots` with high-latency remote · Issue #523 · restic/restic · GitHub . Do you have the cache enabled? When restic has all snapshots cached, then that should be a lot faster. If restic has all snapshots cached, then the performance of the command should be mostly CPU-bound. At least since restic 0.12.0 the snapshots are already loaded in parallel, so there’s not much to optimize in that regard.

1 Like

Hmm yes what I referred was first execution, basically initial creation of cache. This makes clients kinda unusable on first command. Maybe I should think of pre-populating the cache (or maybe some magical shared/network cache might be possible :thinking:).

But sounds like I need to wait for repository format 3 :sweat_smile: Thanks for the info, I’ll dig into issues a bit.