Rotating External Backup Drives in and out

I plan to have 2 external backup drives (maybe add a 3rd later) to snapshot my local machine to one of them periodically. Once in a while (every 7-30 days) I would remove the disk and swap in another one. After a while I’d do it again to swap in the first disk again.

I was wondering if I should just initiate the disks separatly as standalone repos (1) or make them share the same repo id (2) or make them share something like chunker parameters or maybe more (3) (like here)?

More details for how I plan to set it up:

  • automatically backup / with root restic-user privileges (~every few hour maybe via systemd timer)
  • automatically backup ~ with user privileges (~every few minutes via (user-scope) systemd timer)
  • automatically backup some remote folders from my NAS (~once a day via systemd timer)
  • additional manual backups maybe sometimes (yes, I’m aware the disks won’t magically share them
  • forget and prune old stuff with policies (once a day maybe? not sure, yet)
  • I don’t plan to every make the drives see each other connected to the same pc after initial setup
  • I have not setup strategy yet; the Recipes category looks very promising! If you have a opinion on what to use or just do it from scratch, please feel free to share what you think could work well

I guess, no one knows?

Making your repositories share chunker parameters won’t hurt but doesn’t provide you a benefit as long as you are not copying snapshots between your repositories.

The only benefit I can think of when sharing repository ID is saving disk space by having only 1 restic cache directory (which can grow quite large depending on your data). But it won’t hurt either I guess.

Is this critical or just personal data that you are backing up? Can you afford to loose 30 days of data?

Hi, thank you for your answer!

It brings up some new questions though.
Will the failing cache invalidation prevent some data from being saved bacause it thinks it is already on the disk (but isn’t) after swapping disks?
Is that something that can be changed later without copying the entire data back and forth?

Short answer: yes, recent data is not that critical
Long answer: Very valid question and it’s very dependent on the threat model and use-case. I intend to move one of the disks outside the house in case of a fire/power surge/robbery/etc. I’m not sure yet how willing I am to do this every week.
I have two types of data and use cases:

  • documents/photos that don’t change that often; these I never want to loose but the backup doesn’t require to be updated that often. After saving a bunch of new photos I might do a backup and the swap the disks.
  • stuff I work on and that changes constantly; very frequent snapshots should prepare me for accidental deletes, etc.; it would still hurt to loose 30 days but it would be okay-ish the main use-case here is having a “safety/stupidity net”

This really combines two use-cases: off-site + fast, regular snapshots
If I want to make this more robust against failure of regular disk AND backup disk, I could add another on-site disk which would be easier to swap (2 on-site, 1 off-site). For now I think is will suffice

While I can’t give you a definitive answer to this question (better wait for a developer to chime in) you can always delete the cache or run restic backup --force to make sure you are not missing any files when swapping disks.

One of our customers bought a NAS which he brings to office every morning to make backups and carries it home every evening. I would never do this myself but if it works for him … :wink:

What about using a cloud provider to save the critical parts of your data?

Let’s rewind here for a second.

Why would you run around swapping disks, which will most likely put you in a situation where if your house burns down, you lose X number of days worth of data (since you may not have swapped the disks right before the fire)?

The proper solution to your (very valid concern) of the house possibly burning to the ground is to have one of multiple backups offsite. Simply have at least two backups - one that you make to your local physical disk, and one that you make to a server or some other storage on the Internet. Run them in parallel, whenever you want. This way you will always have up to date backups to access in case of a major catastrophy.

I back my stuff up to a local file server and to external servers using rest-server, but you can use e.g. S3 or the cheaper Wasabi for your offline repository if you don’t want to rent e.g. a cheap VPS as hetzner.cloud with their block storage (both of which is great IME). There’s a ton of options, especially with the rclone backend you have so many backends and providers to choose from.

I don’t understand why you are trying to solve this in a more complicated and way less safe way, and I can’t find any reason for it in the text of this thread either.

First of all, I do consider backing up to S3 as well. Also, the append mode for rest-server seems very attractive as a kind of off-line and off-site alternative to unplugging devices, but I don’t have the infrastructure for that at the moment.

It’s unlikely that some kind of attacker would encrypt/delete my local machine, my backup hard drive, and the S3 bucket but it would be possible. I think it might be possible to prevent that with S3 bucket versioning and bucket policies/ACLs, but I don’t know for sure and it seems more complicated.
rest-server in append mode would be perfect for this kind of paranoia with the kind of off-line approach if I had a server/VPS.
Unfortunately, both S3 or a VPS cost money and not just once.

Funnily my NAS died (not the disks) shortly after doing a manual backup of it and running the first restic backup. The disks seem mostly fine except for some btrfs tree corruption – but that’s a story for another day.

This actually made me consider the need for an on-premise NAS vs a VPS for running more services than just backup. So my capabilities and views on backup might soon change. This of course brings up the question for VPS backup… *sigh*

Let’s hope my house doesn’t burn to the ground until I have a final solution. :wink:

Yes, it does cost money. But it’s your data we’re talking about here. Just look at the low prices you get at hetzner.cloud for the smallest VPS and however much block storage you get (which you can size up and down as you wish - start small and increase when needed). Or maybe netcup.eu (which I’ve heard good about from others) might be cheaper with their VPS and Storage offerings. Personally I can recommend Hetzner, they store your stuff on CEPH as well, FWTW.

If VPS is to expensive for you, you could get a cheap computer (raspberry?) acting as a rest-server (or maybe ssh+rclone+append-only) and use it to access your S3 repository.
If you feel really paranoid (as I was at work…) your could have two backup repos, and make backups even days to one, and odd ones to the other.

Unless you’re being targeted by a very sophisticated and resourceful opponent, this is not something to worry about. Even if that is something of real concern, why would you swap the drives? Surely it would make more sense to periodically mirror the contents of one to the other - retaining a single line of snapshots, but stored in two places.

I spoke to some colleagues with servers (at hetzner) who offered some space on their hypervisor for sharing the costs. Which sounds quite reasonable. I’m going to keep my rotation idea though (different repo id, same chunking params) – in case my colleague goes rogue and burns down my house :wink:

Your backups doesn’t seem to be worth much to you from what I can tell :wink:

Again, this is very dependent on the data, how rapidly it changes and how much can be repeated.
I think 4 locations are enough.

I think our main concern is that your proposed solution sounds like a bad mixture of overkill and not enough.

It should be said, that I am deeply thankful for all your inputs and advice! Maybe I underestimate the capabilities of restic: how fast it is, how little traffic and how few requests it uses, etc.
It’s currently hard for me to gage how a backup to a remote location will run and with a local ssd I can be sure it performs as well as I expect. Moving the backup off-site is the next stage if you don’t have a remote target.