This is less a question rather than some thought sharing.
A few days ago, AWS announced a new S3 tier, called “Glacier Deep Archive”. As far as I can tell, it takes the existing Glacier product properties even further, most notably in terms of time-to-restore (comparison) and pricing. At $0.001 per GB-month, storage is an order of magnitude cheaper than what competitors such as B2 offer (restore prices are a different story, though). The downside, though, is that with restore times of up to several hours data isn’t available interactively.
Even if only the
/data subdir is moved to Glacier, restic wouldn’t be usable for much more than just adding additional backups. For everything else (such as prune) data would need to be moved back to a standard S3 storage class first, which only makes sense after Glacier’s minimum storage duration of 180 days and costs the same as 2.5 months worth of storage. I see two possible scenarios based on restic’s current feature set:
- Append-only backups. Given low storage prices you’d just keep old data, even more so if data gets rarely deleted.
- A restore every > 180 days, prune, then move back. Ideally, you’d only restore those packs which were written > 180 days ago and then prune snapshots older than 180 days. Restic shouldn’t need access to any files younger than that, I haven’t tested this, though.
Possible scenarios based on extensions of restic:
- Restic could implement some sort of “lightweight” prune where it never validates or re-packs any existing data and only performs delete operations on packs where 100% of the contained blobs aren’t needed any more. Although this would result in some overhead (partially required packs), it would still free up some space compared to 1).
- An optimization based on the combination of 3) and 2): First, restic would “lightweight-prune” superfluous packs, then only restore the packs which are left && needed, perform a “proper” prune (incl. re-packing) and then move data back to the Glacier storage tier.
I think 1) or 2) might be valid approaches to consider for secondary backups where you expect to never need them. 3) and 4) probably wouldn’t warrant the effort – most likely there are better alternatives for such use cases. I’d be interested to know your thoughts on how this new offer could be leveraged in combination with restic!