Use ext. attributes when looking for changes

[robvalca]

Hi all,
We want to work in a feature for restic but I wanted to cross-check with the experts if this makes sense or not . This comes as a follow-up of this thread:

Backup using cephfs extended recursive attributes

Hi all, First, I want to congratulate all people that makes restic possible! (it’s awesome!) I would like to share my use case so you could advise me. We are evaluating restic for backing up a huge volume of data (hundred of millions of files, hundreds of TB in size) placed in cephfs and backing it up in our S3 service. Ceph folders have recursive attributes that tells you, for example, the time stamp of the last modification of any file or folder under the selected folder. Our idea was to ta…

The idea is to take advantage of file systems with recursive mtime via the extended attributes. Cephfs already have that feature and also our in-house EOS storage system. The idea would be to add a new optional argument to restic backup to allow specifying a custom extended attribute name that restic would use for compare and eventually discard full branches.

As restic already backups the extended attributes, (in my mind) seems that it’s only a question of adding a new ~if but checking this specific attribute.

Do you think that this is doable/makes sense? We are already quite happy with restic but having this feature in the file system makes me think that we could take benefit of it.

Thanks!
Roberto

[fd0]

When evaluating whether or not a feature makes sense for restic to have, we need to balance the complexity that (inevitably) comes with it, the benefit users have from it, and the maintenance costs of keeping it running.

How widespread is this, keeping another (possibly recursive) mtime in extended attributes? Is there a standard for it? Or does every system/filesystem/storage device implements this on its own?

[robvalca]

Thanks for the reply! I understand your point, it’s true than currently every storage system implements this on its own, afaik there is no standard and probably not too many users will get a benefit from it (apart from cephfs users). Just wanted to know if this kind of modification is something more or less doable or something really complex.

Thanks!