Backup tamper prevention with Backblaze or other cloud storage (hacking ransomware etc.)

I’m investigating replacing an existing backup server (non-restic based) with a cloud storage (and restic).

One of the features of the existing server is that it uses a pull model so that if the machines which are being backed up get compromised, the backups (which are versioned) can still be trusted.

I’m aware of the --append-only flag for the REST server, but wanted to look at options for using cloud storage.

I’ve not used Backblaze, but I noticed that it supported application keys, so that you could create a key without delete permissions.

https://www.backblaze.com/b2/docs/application_keys.html

Would it be feasible to create an application key for each server with write-only permissions, and have a separate secured server (e.g. one which requires physical access to log in) perform the prune, and check operations (and maybe make that the only option for restores) by creating an extra application key and granting it full permissions over the restic b2 bucket which the “main” server backs up to?

Would it also be possible to share a bucket between multiple servers and use application key permissions prevent compromised servers reading data which other servers have backed up? I’m assuming metadata needs to be read by each restic client, so that this might not be possible (however application keys can be limited to specifc file prefixes, so maybe it could be made to work?).

Hello @TimSmall,

If I understand you correctly, your plan is to have the individual machines, which are now "clients"of that backup server, back themselves up to the cloud directly, correct?

What I am doing here is, I have a dedicated backup server that pulls the backups from the other “clients” in the network into its directly attached storage, and then uploads those backups to the cloud using restic. This way, you are entirely safe as your keys only have to reside on that backup server – and you have the backup server already, right? So why not just have it back the clients’ backups up to the cloud using restic?

Just my $0.02…

Cheers,
– Durval.

In theory, yes. However, note that B2 doesn’t have a way to create a key that cannot overwrite existing objects. Therefore, for this to work, your bucket must have versioning turned on, and you must have a way to restore the bucket to a point-in-time based on B2’s versioning.

No. Backup data is deduplicated among all of the clients who use the same repository, so there is no concept of who “owns” data in a repository. One pack may contain a blob that is referenced by multiple clients’ snapshots and there is no way for B2’s permissions layer to know this – and you can’t apply permissions to a region of a file anyway. (Actually, I don’t think B2 can do per-file permissions at all…)

Hi Chris,

Thanks for the response!

I assume you’d also need to add something to remove stale files at prune-time (both data and metadata), but this doesn’t look too hard, and the same tool could probably be used to delete newer data/metadata as well.

Alternatively those capabilities could be added to Restic’s B2 support so that the hardened “non append-only” server could carry out these operations (“restore using B2 repo as it was in the past”, and “prune old file versions”).

I don’t know, but the following is a quote from https://www.backblaze.com/b2/docs/application_keys.html -

“Application keys can also be restricted to the files in a bucket matching a file name prefix.”

… so that may or may not be suitable. e.g. an application key could be created which would allow the repo metadata to be read, but not the file contents using the “matching a file name prefix”, if this is the case, then an attacker would be limited to viewing metadata for other servers, but wouldn’t be able to read any of the files. Maybe. Possibly only with patches to Restic’s B2 support.

I will assume that for max security it will be necessary to have one bucket per server, I don’t think this particular use-case will save too much from having deduplication available anyway.

Tim.

No, this is not possible. As I mentioned before, a single pack can contain data that is shared by multiple servers, and even exclusive to a single server. Therefore, this isn’t possible – you’d need some way to restrict access to regions of a single file. Even if that were possible, there is no common prefix for “stuff from client A” vs. “stuff from client B.” You’d need to maintain an ever-changing list of which files are related to which servers and be constantly updating the application keys, assuming B2 even allows multiple prefixes for a single key.