Restoring files directly into a single compressed archive


#1

Is there any discussion about a way to restore files to a single .zip or .tar.gz file?

Am willing to write code. :slight_smile:

I could envision a couple ways of this happening:

  • Direct support implemented into restic’s CLI, with a flag such as --archive zip or --archive tgz or something.
  • The restore functions exposed as a library so that an io.Reader can be returned such that I can pipe the contents of the restore into a zipper or targz-er function. Ideally avoiding writing a whole archive to disk at once.

The goal is to stream the contents of the restore out to some client over a network, while preserving disk space and memory.

This is a pretty niche need, so, I am of course willing to write code, even if it’s external to restic if it wouldn’t be a good fit for restic itself.

What do you think?


Snapshotting a directory from STDIN?
#2

Hi @matt you can already achieve this in at least two ways:

  1. Mount the restic repo at a file system with restic mount and then zip up the snapshot folder you want, with any filters you want, to a zip file.

  2. Mount a zip file as a fuse filesystem and then restore with restic into that flle system. Unmount the filesystem when finished and you have the zip file. (https://bitbucket.org/agalanin/fuse-zip)

Both method avoid ever restoring all the files to a real local file system that takes up space. In both cases the files go straight to a zip.

If you want to stream the zip file directly out of restic to e.g. some sftp location, then you will still need to cut some code I think.


#3

I don’t know much but maybe at some point the restored snapshot or file would be in a tmp folder in order to create the compressed file and then it’ll need to delete the restored files leaving just the compressed one in order to accomplish this. I think using mount option and tar is the simplest way. Once the directory is mounted you just need to point tar to the snapshot you want to save.


#4

Thanks both.

This is an interesting possibility. I don’t love the idea of trusting a fuse mount since they tend to be unreliable and/or slow, but I’ll look into how viable this is for our needs. Still preferable would be for restic to have some native support for this, rather than relying on a fake file system, but if it does the job (even if it’s a little slow), I can work with that.

Thanks for the idea!


#5

Correct me if I am wrong

Restic mount doesn’t work on windows…


#6

I don’t think WSL supports fuse yet. You can vote for the feature here:

You can use a Linux VM or Linux docker container on windows and do it in there, with the output ZIP going to a mounted Windows filesystem.


#7

Instead of implementing a compression directly into restic, why not simply support a tar output stream, so it can be piped to other compressors on the shell?


Snapshotting a directory from STDIN?