First, I want to congratulate all people that makes restic possible! (it’s awesome!)
I would like to share my use case so you could advise me. We are evaluating restic for backing up a huge volume of data (hundred of millions of files, hundreds of TB in size) placed in cephfs and backing it up in our S3 service.
Ceph folders have recursive attributes that tells you, for example, the time stamp of the last modification of any file or folder under the selected folder. Our idea was to take advantage of this information and discard unmodified full branches of the directory tree while going down into the depth levels to speed up the backup process.
The first approach was to develop a tool to traverse the folder and create a file with the list of the subfolders and files to backup and then pass this file to restic using the parameter --files-from. This worked fine and the backup speed improved dramatically. The problem is that when mounting or restoring the snapshot only shows/restores the files/folders backed up in that snapshot, not the incremental view of the folder.
After that, I tried an opposite approach, creating an exclude file and sending it to restic using the --exclude-from parameter but I have exactly the same problem. This approach is better cause when I do
restic snapshots the snapshot path is showed like a normal snapshot while in the first approach you see all the files in a block view making the things more confused. I also tried with the --parent option but seems that it does not work as I expected.
Is this behavior with the --files-from / --exclude-from expected?
Any suggestion about any other solution to this specific use case? Could be this a feature? for what is exactly the --parent option?
Thank you in advance!!