Backing up folder with rotated files?

Getting Help
1

I’m impressed with what restic offers out of the box: encryption, compression (as pointed out below, no compression yet), support for popular cloud providers – god bless you, the developers (I mean it).

I’m evaluating restic to use for backing up a folder with files that are being rotated – the logs. In this folder I’d have up to 10 files, 1Gb each.

Right now this is solved using a the following script (wrapped in a watch -n 3 ./backup_logs.sh line):

gcloud auth activate-service-account \
  my-service-account@my-project-id..iam.gserviceaccount.com \
  --key-file=/path/to/key.json

gsutil -m rsync -r -J \
  /path/to/logs \
  gs://my-bucket

The script is simple enough as is, but that’s the reason I’d like to try and migrate it to restic: I’d like to start small in my endeavour of adopting restic.

Upon initial evaluation, as far as I understand, restic is based on snapshots. Hence, at any particular point in time a snapshot will contain “current situation” in a given folder, e.g. in my case only ten 1Gb files. I’d like to, however, have a continuous (well, maybe once in 30 seconds using watch -n 30) stream of files to a remote storage.

Am I right that for a particular task at hand restic is not suitable?

2

While restic is probably capable of what you want it to do, I personally think that (from what I understood) restic is not the right tool for it. If you need continuous backups for logs you can use a syslog-server and maybe backup from there.

PS: There already was some discussion about continuous backups on the forum.

3

I’d say it’s not what restic is built for, but you can use it for that. You’ll get a snapshot for each time you run restic backup with the files in it which were stored in the directory at that time.

Did you discover the fuse mount yet? You can browse and read the files there, it’ll contain a directory for each snapshot, and you can use your normal shell tools…

Thanks, although that’s a bit too much praise because we don’t support compression yet :slight_smile:

4

Thanks, although that’s a bit too much praise because we don’t support compression yet :slight_smile:

Oh, it is only now I realized this! Sorry, my mistake :slight_smile: Still, restic is a great piece of software even without compression for now. Thank you for it.