I found it’s very easy to make a tag in restic, but it seems almost impossible at present to find them. It appears you need to know the exact spelling of a tag ahead of time. I want a way of specifying a glob pattern or a regular expression to look for tags which match. This would be useful for managing contextually related but different backups. For example, suppose I make 3 alternative edits to a picture. I want to keep backups for all three of them, but in the end I need to choose just one picture. So, I want to make three related backups which I would like to be able to tag with experiment-A, experiment-B, and experiment-C. Then I want to find all of the backups related with this experiment. I wish I could do that with:
restic snapshots --tag 'experiment*'
In my actual use case, I’m not just interested in modifying one picture, but in making controlled variations of many pictures in a directory structure and I would rather not mutate them in a way which would garble the file names with additional naming conventions like adding version suffixes.
At the very least I want a way to list all the tags currently used in the repository. Is there a way to at least do that? I would have thought for sure there would be something like:
restic tag --list
restic tag --list pattern
Of course, maybe the answer is that I just need to figure out how to use git for managing my many gigabytes of data. It just seems like restic is so close to making this convenient enough without having to deal with the rest of the git feature set.
I don’t mean to dismiss your wishes, but I must say that the use case(s) you describe here are a pretty perfect match for version control such as Git, rather than backup such as restic. Honestly you seem to be using the wrong tool for the job
Using Git or similar enables you to do more things, such as quickly switch between your different versions, as well as keeping different versions of collections of objects, and much more. I guess you already know that. And there isn’t a lot of Git commands you need to do this, perhaps you just haven’t gotten into it enough yet
The core challenge I see with git is that it’s really tailored to being a distributed source code management tool designed to be used by many independent contributors. Because of the extra requirements necessary to work well for that application, managing hundreds of gigabytes of arbitrary files is hard for it and often creates a lot of pointless work messing around with .gitattributes and setting up remotes. Furthermore, there is a really terrible encryption story around git managed data. Yet, that’s really important when working with this amount of data because you’ll want to archive it somewhere. People keep trying to bolt on things like git-lfs or git-annex to work around some of these problems, but this task is unrelated to managing source files and that’s git’s primary purpose. I tried to use git annex, but it left too much of a mess for me and I don’t trust all of its wonky Haskell parts to keep working for long after Joey gets bored with it. I looked at git lfs, but ruled it out because it doesn’t seem in any way a simple solution. Restic, with its large community and being implemented in go seems like a much safer bet. Also, I am already invested in the tool for backups so I have some motivation to remember how to use it. I think all it really needs is richer tooling around tagging.
Indeed, if you have that much heavy material Git is not made for that. I wonder what other version control software there is that is actually made for large objects
May I ask why you don’t just make copies of the file you’re working on, keeping one copy per “version” for the time that you work on it? You seem to have the space and that’s a very simple workflow.
I think the main thing I am attracted to restic for is keeping track of my experiments in a routine way that fits within my other routines. Right now it is very common that my other routines are very sensitive to the paths which are used by default. Changing the paths could cause additional problems which I would need to control for in my experiments and that control would require a lot more custom work than I am aiming for. I would rather restore the whole directory tree than have to change the path.
Relatedly and more concretely, I am a huge fan of the restic mount feature since it essentially gives me time traveling capabilities across my snapshots. That works really well for me. It is far more convenient than checking out branches with git. To the extent that it can become easier to filter down my “snapshot completions” to those with the right tags, hosts, and paths the easier it is for me to steer my time machine.
@GuitarBilly Thanks. That was my conclusion as well and the only way I could move forward in the short term. So, thanks for getting that down on paper so it might help someone else. I personally would prefer not to have to maintain a janky shell wrapper in the long term since I am mostly only proficient in using tools which are far less stable than restic and this functionality seems to me like something which could benefit a lot of users by significantly extending the ease of use and applicability of tags. Git style globs are already available for paths, so not having it also for tags was surprising to me.
