I was reading this issue #1380 and got thinking… generally, most users would probably not want to include these things in their backup:
restic’s temporary files
restic’s cache(s)
the restic repository being backed up to
… and people would probably prefer to exclude those from a backup.
Does this sound right? If so, I’d be willing to look into patching restic to ignore these by default. Or perhaps some discussion about how to obtain these paths in the code of the restic backup command (cmd_backup.go).
(Of course, if these are excluded by default I’m not sure how to force-include them, except perhaps to explicitly list them in the args of the restic backup command.)
I think most users would not backup cache and temporary files… I don’t do it because I think it’s a waste of space for my use but if you want to restore your home directory exactly as it were then maybe you’ll need those files too. For example, I don’t really understand why some people are preoccupied to backup thumbnails but they do. The multiple times that I have to do a restore I just do it for the files that I really need. I backup all my home directory but when I perform a restore I usually ignore a lot of folders that I don’t really need. I prefer to do all the configurations again so I don’t have an overcharged system or a bloated home directory with folders that I don’t really need. Personally, I don’t know how good it would be to exclude some files by default but for data-hoarders it will not be a good idea. At the end, when it comes to backups I think it’s better to have some extra files in case you need them in the future.
Mhh I thought restic would already exclude its cache from the instance its currently running when, for example, you’re backing up ~.
Or did I miss something?
Just out of curiosity: what benefits would a user have when backing up restics cache? I might just not see the use case here.
Well, as I understand it, restic’s cache is only ignored if the CACHEDIR.TAG file is present (and it has a signature inside it, I believe). It’s not explicitly listed in the --exclude list you pass into backup.
But yeah, I agree with both of you: that backing up those three things is not necessary – that’s why I’d like to discuss having them excluded by default, or an easy way to exclude them. (Imagine backing up / – the root of the system – which obviously includes these things.)
Yes, it does. We should work towards this, in my opinion. The temporary files for the current backup process must never be saved (that leads to nothing sensible), on Linux it’s not an issue because the temp files are deleted right after creation, so they don’t appear in the file system at all. Also, the current repository should never be saved to itself. It’s the same with the cache directory for the current repository, but that is only one directory within ~/.cache/restic.
What I also like to change is moving all the different exclude functions from package cmd/restic to e.g. internal/exclude and adding more tests.
I think there’s a fundamental misunderstanding here. Those of us that happen to back up thumbnail files don’t do that because we sit and spend our time making sure these files are included. Instead, and on the contrary, we did not spend our time actively making sure to exclude these files, because our time is more valuable than that compared to how little space they will occupy in the backup repository.
This makes even more sense when you consider the fact that no matter how carefully you craft your excludes, you will always have files in your backups that you would want excluded if you were given the choice (there’s always files you don’t think of and that pop up before or after you crafted your excludes). So, in summary, we’re not preoccupied, we just have better things to spend our time on than excluding peanuts
Personally, the main reason I don’t write exclude rules for small files, even if they are many, is that as I said, there will always be files I missed. If I can’t do it properly, and it just costs a little more space but in the big picture hardly any compared to how much the rest of my backup data consumes, then why bother.
Same here. Just because you happen to have some excess backed up doesn’t mean you have to restore this excess.
