Wasabi Compliance bucket and lockfiles

Like others, I’m looking for a way to back up to a cloud provider and have some assurance that the files don’t get overwritten or tampered with for a period of time (I don’t want a fully append-only setup as I do want to expire backups after a certain age to reclaim space). We’re using Wasabi at the moment as our backend.

Wasabi has a “compliance” setting that prevents files from being deleted for a configurable number of days. Unfortunately, this policy is bucket-wide, and once set it prevents restic from cleaning up its lock files (because nothing can be deleted until its age is greater than the policy limit).

Someone hacked around this by using rclone:

The approach is to use two buckets: one for data (with the retention policy applied) and one for locks (with no such policy). They set up two rclone instances (one for each bucket) and then use a proxy to transparently redirect the “lock” requests to the instance that doesn’t enforce a retention policy.

To avoid all of that, could a feature be added to restic where one can override the storage system for locks specifically? In the most minimal case, just specifying a different bucket would work (all other params would stay the same), but for maximum flexibility allowing the user to specify a full second storage system would be ideal (they could revert to a local file system or another provider if desired). Finally, exposing this functionality via environment variables would be great so we can bake it into scripts.

Thanks!

I’d love to see restic integrate Redsync as an alternative locking system. Particularly in server environments, it’s not that difficult to stand up a Redis cluster – and if you already have one, you’re all set. The “file exists” locking system in restic has always seemed a bit fragile to me, due to the “eventual consistency” models of object storage services.

1 Like

Or may be it is possible to add possibility to setup lock directory absolutely separated from the rest of the repo?
something like restic init --repo /srv/restic-repo --lock-path s3:s3.amazonaws.com/bucket_name

Related question:

Does enabling file versioning on the cloud storage provider address your compliance needs? (i.e. the ability to retrieve the repository as it was on a particular day)

Regards.