I currently backup various machines to a local server using the restic ssh backend. The server is raid1 and does daily rsync of the restic directory to a backup drive. The current restic volume is about 600GB.
I have been also backing up the restic directory to Code 42. This worked great but in the last year, it has been lagging miserably. It’s $120/year.
So, I’ve been thinking about ditching Code 42 in favor of Backblaze B2. I know that Backblaze usage has been discussed on and off in this forum, but I’m still unsure of the best strategy for cost and efficiency.
Given the business model for B2, my sense is that I’d better off doing an rclone of the current restic data to Backblaze periodically (e.g. once or a few times per week, maybe up to once a day) rather than using the rclone backend explicitly. Then I’m simply adding the new packs, deleting old packs and replacing the index.
Do people who have tried this have general comments on best practices? Is it better to do the rclone of the restic directory or use the rclone backend directly? Thoughts? TIA!
One thing to consider is that if your repo is somehow broken, corrupted etc. then using rclone to push that same repo to Backblaze will simply propagate the broken repo to your secondary backup, so it’s not really a “second backup” so much as a “backup of your backup”. I would probably opt for two separate backup runs and keep your concerns separate between the two endpoints receiving your backups.
Yes, I see your point. I’m currently sending backups from 3 machines to a local server via ssh to take advantage of deduplication (the 3 machines have many files in common).
I could do the same with the Backblaze B2 backend I suppose, maybe less frequently. That’s a bit more complicated in some sense but doable. I could simply add new timer scripts.
Any thoughts on what strategy is best given the Backblaze B2 cost structure (transaction types, etc.)?
With backblaze you never really hit transaction type limits doing just basic backup, at least with restic. All you have to look out for is restore bandwidth fees, and even that is reasonable.
And also you probably want to make sure versioning is turned off at the bucket level - it won’t “hurt”, but will cost you a good bit of storage space ($) without benefit (my understanding).
I have a similar setup: local backups to an HDD and then another repo on b2. I thought about using rclone to copy the local backup files first, but for reasons above decided against it.
Have considered restic copy? It will copy over some/all snapshots from one repo to another. If you keep the chunking params the same, it’s very efficient.
I have a Backblaze bucket for Restic about 400Gb and it’s costing me just under $2 US per month. Highly recommend!
Using btrbk (all btrfs filesystems) gives me quick hourly/weekly/monthly local only snapshots.
Restic is only to backup to B2 and I run it directly on the filesystem, not on the btrbk snapshots. I run forget->backup daily, and prune->check weekly with zero issues. Systemd targets and Before/After ensure sequential operations so I never have to deal with locking issues.
A typical nightly backup, adding ~600 Mb to the bucket takes about 10-15 minutes. The backup + prune and check on Sunday night takes around 30-40 minutes. My /var/cache/restic folder is about 1.3 Gb.
I would be interested in a similar setup only slightly different. I have 5TB to store and would like to keep that on disk like a full backup, and backup al increments to B2.
in the case of recovery i’ll download everything from b2 onto the disk and go from there.
I could simulte this with duplicity but it’s deduplication is not that good for my usecase so i want to use restic instead. Any thoughts if the offline disk + b2 would be an option?
First of all: Welcome to the Restic community!
Now to your question: Loads of folks do exactly this. They do a backup of a system to a local drive or NAS and then use some other means to sync this repository to a cloud storage. You could use rsync, rclone or other software to synchronize the repo.
Hi, that’s exactly why it’s different because I don’t want the disk to be connected. So backup fully once, store it (underground for all I care) and backup changes to the cloud. But calculating the changes requires the disk to be online right? But it shouldn’t be a requirement because if all the cashed data on my computer… Hope this clarifies my question somewhat.
restic does not do incremental backups in the traditional use of that term. To simplify (overly) each snapshot is a picture of your file system at that point in time. The picture identifies which blocks of data are required to reassemble your files at that point in time. These blocks, because of deduplication, are not necessarily stored contiguously. moritzdietz answer is the correct one.
However, at the expense of storage space, you can do a restic backup, then rsync/rclone or other wise copy that backup to the media you are going to bury, and then continue on backing up periodically. Then you can occasionally dig up the buried backup, update it against your current backup and then bury it again.
I just want to understand. If i backup to b2 everything is up there. But only the indexes are required to calculate what to backup in the following backups. IIRC from the docs, these indexes are stored locally as to not download everything over and over again from b2. And, if i’m not mistaken, blocks (once written) will not be updated during simple backup activity. New blocks will be added (both index and data) but no block will be changed.
Am i wrong in assuming that once stuff is uploaded and the indexes are locally cached, i could be possible to save new indexes and data blocks to other storage?
I’m asking because 5TB of B2 storage is costly and takes a long time to upload. I had hoped to get around that issue.
I can of course, by using a remote PC with a 6TB disk and have an airgapped offsite backup. Which does include a running system, (with HDD’s booting and slowly going bad over time), the possibility of attacks on the SSH machine, electricity costs, noise etc.
