Differential restore of big files

Hi @all.

i try to replace my self-made backup-solution for LVM-based virtual machines with restic. Nearly everything runs very well, but i need a solution for restoring Images over slow connections.

When i restore a file with restic-mount, it will transfer all the file over my slow connection. But similar to backup there is only the need to transfer the difference from the restic-snapshot to current LV.

I think about an option/command like

restic -r sftp:root@10.10.2.2:/srv/restic1 restore /dev/vg0/test_vda --target /dev/vg0/test_vda_restored --restoredifffrom /dev/vg0/test_vda

So restic could compare if a block already exists identical from local ‘/dev/vg0/test_vda’ and use that. If this block doesnt exist restic can use and transfer the block from remote-repos.

If that is completly unrealistic i have to use the restore-part of my old solution that compares local-lv and remote-restic-mounted image and transfers the different blocks.

Or are there any other restic functions which i could use for my case?

Greets from bavaria!
Freisei

If your file sizes match, Make restore resumable by aawsome · Pull Request #3425 · restic/restic · GitHub could be what you want.

The PR reads existing files in the target (if the size matches) and checks for each blob if its contents are already correct in-place. If yes, the blob is not downloaded for restore.

Thank you, Alex!
I think this is the right direction. However i have to test if restic can restore to block-devices.

Do you think its possible to extend this PR to allow a different restore-target (in this case a LVM-Volume) while keeping the mechanism of “use local first”. I think there must be an additional parameter (like suggested above)

Greets from Bavaria, Freisei

Actually I’m not sure if restic can restore directly to a block-device.
If it does, I can extend the PR such that for block devices the size check is omitted.

I would however argument against an additional parameter. Copy/Syncing your local copy to the destination first and then running restic restore would also do the job. Adding this feature would add more complexity to the PR and make it more unlikely to getting merged.

OK, i think i have to open/ask another topic to check if

  • restic can restore directly to a block device
  • redirect the restore to a specific block-device OR
  • restic can follow a link to a block-device when restoring.

If restic can not restore to a block-device i am lost here, i often dont have enough space for a restored-file + the restored-LV because most times i have to keep the “defective” LV.

I can look for that in some days.

Do you think it´s possible to implement your differential-feature in the dump-command? that will maybe solve some of my issues…

Not without making some larger changes to the dump command. Currently the command only outputs data sequentially, but never reads data. It might make more sense to adapt the restore command accordingly. However, unless you can make the changes yourself, don’t expect anything to happen in that regard anytime soon.

Hi Michael,

thanks for your repliy. I can work with my old solution. Would be fine if this feature could be implemented in future - i think there are others using restic for backing up or similar images of virtual machines. I am not a good programmer and don´t know anything about go-Language so it would be nearly impossible to implement by myself.

Is there an official wish-list or should i open an issue at github?

The way to go is to open an issue on github. There’s a feature request template.