Multi Tier storage under a "single repo"

Getting Help
1

As a data hoarder I have a single repo which just keep growing over the years and now reached the 22TB ( my first snapshot in this repo was done with restic v0.8.3 which was the latest at the time ), obviously I have a lot of the data which is super old and very unlikely to be accessed, these old snapshots are the one which I want to move to a cold storage rather then taking space on hot and warm storage spaces, before I do something irrecoverable I wanted to validate my thinking, my idea is the following:

I will use the restic copy command to get all my snapshots over to a cold storage device, and aside of it the restic binaries to ensure that I will be able to restore much later too ( although I am mirroring the GH repo including the releases already anyway which is part of the backup)

restic copy --from-repo hotrepo copy -r /coldstorage1

restic copy --from-repo hotrepo copy -r /coldstorage2

Next I forget the snapshots in the hot storage

restic forget -r hotrepo --host myhost --keep-last 1

restic forget -r hotrepo --host myotherhosts --keep-last 1

and as I create the cold storage repos with “–copy-chunker-params” I can at any point quickly and easily copy back my old snapshots, OR use them directly too.

Relatively simple plan but still wanted to validate the approach and if I am thinking about repos in the right way, is there any better way and what should I watch out for.

1 Like
2

@KissT
few things come to mind after reading your proposal.

  1. can you elaborate why you want to copy to two coldstorage repositories?
  2. I would verify the coldstorage repo(s) with restic check and preferably with restic check --read-data before forgetting snapshots in the hot storage. this will guarantee the data integrity.
  3. since you’ve been using restic for such a long time, I assume your repo can be format v1 (without compression)? if so you can consider to upgrade your future hot storage repo to v2. you could then restic copy that to a new coldstorage3 repo going forward. read about it here: Working with repositories — restic 0.17.3 documentation
  4. after the forget command you should run prune to actually delete data and reduce the repository size. refer to the documentation section here: Removing backup snapshots — restic 0.17.3 documentation
3
  1. Simple due to tolerate HW fault, it could be just dd or rsync too, but as the src of truth is the repo I would prefer to copy from there to reduce cascading errors
  2. Check on the cold repo is a good idea I add it to my flow
  3. I did migrate already to v2 so now I have compression too
  4. Prune was on my radar just forget to call out, also good call.