Using restic to backup encrypted containers?

[tholin]

@fd0

You won’t get to benefit from the deduplication built into restic so much

There shouldn’t be much duplicated data in the container anyway except for unallocated space. I’m more interested in deduplication between snapshots.

@fd0

Another downside of your approach is that you can’t just use the restic mount command to browse around in the backups

That should work. I’ve tested it on a local repository already. I can loopback mount the container from the fuse filesystem. I’ve done the same thing with my current setup over sshfs. It’s very slow but useful in a pinch.

@fd0

The config file is not so important, we can live without it

I though the chunker_polynomial was required but now that I think about it it’s only needed to backup new data, not reading old data.

@fd0

When a file in data/ is damaged, it depends if it’s a metadata file, all files referenced cannot be restored, but in theory the data is still there. If it’s a data file, parts of files won’t be restorable.

I did some testing on a local repo with a single 1GB file backed up. Looking at the content of the snapshot I got the corresponding tree blob id. That tree blob got a list of all the data blobs for the file in order. Using the index file I figured out which pack had the first data blob of the backed up file. I deleted that pack from data/ and tried to do a restore. Restic tried to re-read the missing file several times which isn’t that useful on a local repo. Eventually it gave up without restoring anything. Same thing happened when I corrupted the same pack file.

Restic had access to the tree blob so it knew which data blobs were needed. It also had the length of all data blobs from the index file so it could figure out the right offsets for the data it still had but it still gave up after the first error.

Loss of the tree blob means loss of chunk order so that’s loss of the entire file.

@fd0

If it helps anything, I’m working as a penetration tester, so I get to break systems and crypto for a living. Which does not imply that I’m overly qualified in implementing secure systems, but I have a very good idea what the most relevant vulnerabilities for such systems are and how to prevent them.

I’m mostly worried about mistakes in the implementation. Like that security bug in tarsnap. It used AES in counter mode but after some refactoring of the code the counter nonce wasn’t incremented anymore. The developer of tarsnap is a genius but he can still make mistakes. If there are few contributors those mistakes might not be noticed right away.

@fd0

We cannot guarantee that, it’s very hard to do (depending on the backend). We’re trying very hard, though

Is that only a problem with prune or all operations? Adding new snapshots should only add new files. Those files might end up truncated but on the next run restic could check that old files are fine so the new snapshot won’t reference the old corrupt data.

@fd0

That’s very hard to say. From Europe at least, B2 is very slow due to the high latency of at least 800ms until the HTTP response header arrive for each API call. DigitalOcean Spaces is faster, but I’ve gotten reports from several users that the service seems not to be stable.

Perhaps it’s better to ask which one you use (if any). The setup used by the developer is usually the best tested one.

@fd0

I’ve heard very good things about zfs, did you try that yet?

Zfs itself might be good but I’m using Linux so that means zfsonlinux. That wasn’t so great last time I checked. It’s an out of tree module so you constantly have to rebuild it when the kernel change. Last time I checked it didn’t have trim support which would be useful if I want to use it in the container.