I realize that restic is designed to use many server backends and S3 and B2 are commonly used.
But I can think of advantages to backing up one or more laptops to a local server using sftp and then backing up the resulting repository to cloud storage (for example: “crashplan for small business” which has unlimited storage for $120/year).
Advantages include:
Fast backup times to the local server
No transaction fees (e.g. incurred with B2 and AWS).
Fast restores
And, of course, no storage limit
The main disadvantage seems to be:
If/when the local server fails, the restore time will be very long. However, that restore of the repository (e.g. from crashplan) can run in the background.
I do this, both at home and for the servers I manage at work. Restic backs up to a dedicated backup server running a RAID10 volume, which then syncs with S3 daily. We can do fast restores locally but still have the off-site copy for disaster recovery.
This is not a disadvantage. The alternative without a cloud copy is “when the local server fails, the backups are gone.” A long restore time is better than not being able to restore at all.
To sync to a cloud storage is a very smart move in my opinion. A lot of people here do that using rclone. I don’t think there are disadvantages. If local server fails, you have a copy in a “cloud” service. That’s good.
I do this but I also acquired a 2TiB physical disk to store a copy of the local server repository, just in case the local server is gone, I have another local copy and if that is gone too then I have the “cloud” service copy.
Thanks! This seems like a promising strategy. I’ll try it for a while.
I meant: a disadvantage relative to using the cloud storage as a direct back end. I completely agree with your point above: no way I would trust a single spinning array for long-term backups.
I have been doing something similar here with rsync. But I wonder if this is worthwhile? Perhaps it would be better to not wear out two disks at once, or better yet, do RAID0. E.g. I assume that I would want to get back the most recent copy from the cloud in the case of primary local storage failure . . .
If I understand RAID0 strips data between disks but then you’ll need all disks to have a complete repository. If the there are two disks and the first one fails, then how do you know which part of the repository to restore from the cloud? I don’t think it is impossible but, isn’t that too much of a hassle?
If I understands how restic works with its repositories, restic sees repositories as a sole entity (I don’t know if I’m using the right terms or if I’m right at all, maybe @cdhowie could correct me if I’m wrong) so if the repository is incomplete or divide between two disks then if one disks fails, the repository is incomplete and by result you have a damaged repository. I would prefer to RAID1 so you can mirror everything between the two disks instead of stripping out data between them.
I use rsync too for this task but I’m not just copying the repository but also more data from the server to the external storage. I’m stuck with it for now because I don’t think (or trust) my “server” (Raspberry Pi) to mirror disks… I prefer to do it manually for now.
Oh, my bad! Of course, I meant mirroring which is RAID1. That way, if one disk fails, you have another copy. However, that said, if the controller fails, both disks can end up corrupted. This has happened to me, but not for a long while.
Yeah, I’m using a systemd timer to schedule an rsync, but other than that, it’s the same as your strategy.
That’s the least of your worries… the filesystem structures are toast at that point. In other words, the good disk wouldn’t be usable either. You’d need to start over: destroy the RAID0, replace the failed disk, recreate the RAID0, reformat, and copy the data back.
RAID0 amplifies the failure rate between all disks in the array, as well. The more disks you add to the RAID0 the more likely it is to fail sooner. RAID0 should only be used when you require the performance benefits of striping and can tolerate the higher failure rate, or when you are combining the RAID0 with mirroring at a higher level – though usually you’d use RAID10 in that case.
(For example, I had a 3TB RAID1 and wanted to upgrade to a 6TB RAID1. Instead of buying two 6TB disks, I bought only one. I combined the two 3TB disks into a 6TB RAID0, then built a RAID1 out of the 6TB disk and the 6TB RAID0. I’ve already sustained a failure of one of the 3TB disks because it was mirrored at a higher level. RAID10 would not be useful in this scenario because the disks are not the same size.)
In other words, the good disk wouldn’t be usable either. You’d need to start over: destroy the RAID0, replace the failed disk, recreate the RAID0, reformat, and copy the data back.