New at Restic… I think we’ll be friends very soon :-).
Well… I’ll describe what I’m thinking… surely you can help me here. I’m planning to use Backblaze for a huge photo backup (1.5 Tb). Backblaze gives me a cheap an easy way to backup all, but there is no client side encryption.
Here comes Restic to play. I can upload my restic repository and sync all changes… keeping the low cost and availability of Backblaze… only 1 issue here.
I don’t see the way… in case I miss 1,2 or 10 files from my backup, how can I select from the cloud which restic files are needed in order to restore them (instead of download the full 1.5 Tb).
I know that exist B2 Cloud Storage… that probably suits better… but it’s more expensive.
@zcalusic already wrote how to access a single file from your backup (using restic mount or restic restore). I’d just like to add that restic will only download the data that is necessary to access the file. It’ll be a bit more than the raw file size (depending on how much is already cache in the local cache), but it’ll for sure not download the whole 1.5Tb
Let me explain myself a bit better. I have a huge amount of data that I need to access to fast from my computer. I want to store a secure encrypted copy in the cloud, but has to by client side encryption only (I don’t want to share my symmetric with anyone).
Well… In order to do that, It’s fine for me create a local copy of my 1.5 Tb of data, on another HDD, but this time encrypted. Then, I can use any cloud backup in order to store my data safety.
Well… this is ok. The main problem is… let’s suppose that I need only 1 file, not the whole 1.5 Tb, bun only 1 specific file (let’s suppose that I don’t have it anymore in clear, neither encrypted… but Backblaze I.E keeps copies of deleted files 1 month… therefore I can recover)…
Here comes the issue. Using Restic, I don’t have any way to recognise my file. In Backblaze, I have a bunch of files that doesn’t correspond with the filename and location that I know I have on my unencrypted copy.
Now… my options are:
Create a batch that encrypts on AES256 and keeps the filename and folder structure (aescrypt)… I.E…
If I use Restic (that does everything for me… ) I need to know which files of the encrypted backup i need for recovering my file from Backblaze, for this particular file only.
On both cases, when I know the encrypted backup file(s), I can download it, and restore it… instead of download the whole 1.5 Tb of data.
I hope I clarified my newbie question now.
Thanks a lot.
PD: in terms of security… password derivation, salt, …(I mean the way that restic and AEScrypt encrypts)… is Restic more robust than AEScrypt?
Yes, it will complain loudly, you can then delete the files and download the (hopefully) pristine ones from the service.
@p314 I’m not sure if restic is the right tool for this particular job. It’ll cut files into smaller blobs, and recombine them into files again, which are then stored on the local disc in the repo (which you then plan to sync to the clolud service). There may be many files needed in order to restore a particular file. The easiest would be to use a service which restic has support for, either directly or via rclone. Then you could just use restic to downloaded the needed files for recovery.
It sounds like an encrypting file system (encfs, gocryptfs, or something similar) may be better suited for your use case. There you’d have a 1-to-1 correspondence between source files and encrypted files.
In order to find out I’d suggest trying restic (and anything else that may work for you) on a small subset of the data. Then simulate all the scenarios that you’re interested in. Shutdown your machine, and try to restore data on a completely different machine for example. Or delete a file (or move it to a different directory) and try to restore that. It’s the only way to find out if it’s going to work for you
You could do that with restic, but that’s not the way it was intended to be used. Most of the code assumes that files could be downloaded from the storage location on demand, and will fail if something is missing (that you need to place there by hand, a manual and labor intensive process).
If you can get restic to access the remote location directly, it’ll do that for you. It’ll only download the data needed to restore that single file.
I don’t know what AEScrypt does, but we try to make restic as secure as possible. Here’s a short review of the crypto code and design, in case you’re interested in such things: https://blog.filippo.io/restic-cryptography/
In this case I can recommend saving the data into a repo locally (if you have the space) and work with that directly, but also sync it (including file deletes) to B2. If disaster strikes and you need to restore something, then you can either use the local repository (if available) or the repository stored at B2 directly via restic. This way, it’ll only download what’s needed to restore the data in question.
Hi… thanks for the response. This solution sounds good… but as I said before…B2 vs common Backblaze is 2x pricing… and I need to store data 2 times. If I go this way, I’ll consider use Rclone. I saw that I can perform backups directly to B2 using Rclone, and I’ll remove the step of local sync the restic backup repository.
If I pretend to not pay the 2x for B2… I should need to perform backups on local, and then sync automatically via Backblaze common automated backup.
In case of failure (accidentally delete of my original data)… I have the local copy of the backup. In case of HDD failure… I still have the whole restic stuff in cloud…
The pros of this system is that the bandwidth for upload/download is free… no cost. Also there is no cost for cloud space… therefore I’ll pay same for 1 Gb than 1.5 Tb and growing. I don’t need to care about this
The only disadvantage I see is that I need to download all for a local restore… not only some files…
on B2 option… I would download some files… but I don’t see when do I have to do it since Restic will give me snapshots… and I can recover old files from there … right?
In case of HDD failure … I still have to order all my data from their DC… via HDD…
It’s hard (but not impossible) make the Backblaze just dump files on the encrypted partition… and forget about the rest (my local drive… etc).
I can transparent encrypt files with a password that is stored on Backblaze. Well… I don’t rely on it… but it’s extra add if you make any issue and start to upload things in “clear” (as I said… #1 is not always easy).
Backblaze client doesn’t upload empty folders. Now… I found that I’m not able to restore a backup test I did because I missed the “lock” folder. Since probably if I create this manually will solve the issue, I need to know if Restic will generate empty folders and when.