Shared Metadata Database over multiple computers

Hi there,
New to Restic and I think it is fantastic.
What I would like to is this:
I have an Auto scaling Group accessing a ObjectiveFS S3 bucket filesystem backend that I would like to backup using Restic.
I would like the ASG instances to perform this backup. I would like Systems Manager Run Command / Maintenance Window to accomplish this using tags - i.e. Group / Backup or whatever.
My issue is, that normally if you run a second backup after a first, it completes in seconds, however, if you do this on another server - same source and same destination it starts from scratch which I don’t want.
I am assuming Restic has some sort of database it uses to keep track of backups and restore.
What I am looking to do, if possible, is to store this metadata DB on the shared filesystem, have Restic point to it and then when any of my instances (which can get killed at any time) run a backup it will take seconds and do incremental only.

Is this possible? If so, how.

Let me know.


Sounds like you are looking for the --cache-dir parameter to point restic to a specific local (or in this case remote) cache directory.

OK great - I’ll get on the case.

It sounds more like restic is not able to find the appropriate “parent” snapshot to use. The parent snapshot enables restic to compare metadata only, instead of reading and hashing the contents of every file.

Restic, by default, looks for the most recent snapshot that shares the same hostname and paths as the snapshot being created. If you are not specifying --host when creating the backup, that’s likely the issue, as the IP-based AWS-generated hostname is being used instead.

I would verify that you are setting --host to a reasonable and consistent value for each logical backup set. (Don’t set it to the same thing for different data!)

If this still doesn’t get restic to choose an appropriate parent snapshot, you could parse the output of restic snapshots --json, determine the parent snapshot yourself, and pass the snapshot ID to the backup command using --parent.

Awesome - I added both these answers together and got it working with:

restic backup /data/xnat/archive --tag dailybackup --cache-dir=/data/xnat/restic-cache --host=xnat-production

Used a cache directory on the shared ObjectiveFS file system and created a hostname to use on all instances.