Exclude list for $HOME and stratergy for Blacklist Vs Whitilest

[taekwondo0512]

I did find two post regrarding exclusion list →

Exclusion file / ignore proposal for backup

Hi there, I am looking on some feedback to get a file to use as input for --exclude-file backup option for Linux (Ubuntu in this case). I am not sure if it is required to scape spaces (\ ) in directories or file names (e.g. $HOME/.config/chromium/Default/Application Cache) Here is what I have for the moment for the .resticexclude file. Comments are welcome: $HOME/Downloads/* $HOME/Pictures/* $HOME/Dropbox/* # Maven target target/* $HOME/.Private $HOME/.cache $HOME/.gvfs $HOME/.local/share/g…

Useful exclude patterns

I don’t know if this is appropriate, so if this is not acceptable for the forum remove the message etc. I’ve had to exclude a fair number of files from my home directory as a GNU/Linux desktop user. As far as time investment in restic, I think that developing and debugging the exclude file is the biggest chunk. I’m going to leave my exclude file here. I think that, if there’s interest, it could become a repository of exclude patterns for all sorts of users, perhaps should go on the wiki some d…

But is there any more up-to-date community mainted exclusion list? Can Restic as a project take initiative for this.

Otherwise what’s your stratergy to build up exclusion list? I was thinking Whitelist Everything then individually blacklist. To figure out what to blacklist, i can mount latest snapshot, run qdirstat and see to items count under a directory.

[GuitarBilly]

@taekwondo0512
two questions for you:

1. what operating system [OS] do you run? this makes a huge difference for exclude files and the advice from the community.
2. what is your goal by applying an exclude file?

I know that npbackup has exclude files for all OS it supports here on github
npbackup/excludes at main · netinvent/npbackup · GitHub

Personally I do not like to use somebody else’s lists as it seem it always contains some exclusion that I need to have in my backup (what is waste for one is gold for others).

[taekwondo0512]

what operating system [OS] do you run? this makes a huge difference for exclude files and the advice from the community.

ArchLinux.

what is your goal by applying an exclude file?

1. Reduce backup size: I am mostly limited by low storage space
2. Reduce backup time: for example podman containers, android dev files and so on just increases size and time due to high item count
3. Keeping the backup clean: that is not having unneeded files in it.

Personally I do not like to use somebody else’s lists as it seem it always contains some exclusion that I need to have in my backup (what is waste for one is gold for others).

Yes, that true. But maybe we can have different levels of grouping inside exclusion list, where each levels is categorised by like must, maybe, opinonated and so on.

For time being, i will analyze my backups with QdirStat and use ItemCount to get idea of unneeded backup location and them to exclusion list.

I know that npbackup has exclude files for all OS it supports here on github
npbackup/excludes at main · netinvent/npbackup · GitHub

Still hoping we can have one restic community maintained which has wider reach.

[kapitainsky]

@taekwondo0512

Still hoping we can have one restic community maintained which has wider reach.

I doubt it will ever happen. It would be massive undertaking given number of different operating systems supported (not even going into very different software used) and very fuzzy requirement - beyond maybe not backing up cache directories (CACHEDIR.TAG file can be used for it:)) everything else depends on what user wants e.g. one person might consider logs as important part of their backup but for others it is waste of space.

IMO good strategy is to rebuild your system at least once from scratch and note what was needed (take notes) to restore your preferred functionality. This is exactly data you must have in your backup.

[taekwondo0512]

I doubt it will ever happen. It would be massive undertaking given number of different operating systems supported (not even going into very different software used) and very fuzzy requirement - beyond maybe not backing up cache directories (CACHEDIR.TAG file can be used for it:)) everything else depends on what user wants e.g. one person might consider logs as important part of their backup but for others it is waste of space.

Yeah, it too much opinionated. Too much headache for what to exclude.

IMO good strategy is to rebuild your system at least once from scratch and note what was needed (take notes) to restore your preferred functionality. This is exactly data you must have in your backup.

Yes, i will try this in future. Thanks.

[kapitainsky]

Also value your time. Storage is cheap compared to it. Instead of pursuing never ending quest for perfect exclusion list it is better to backup a bit too much. I exclude biggest space offenders and ignore the rest. 80/20 rule. Like always.

My personal approach is to have two backup sets. One contains all my personal data (Docs/Pics etc.). This is long term backup - almost archive like. I take time to do it very right and have multiple destinations (including offline) where it lives.

The second one is short lived (e.g. 3 months worth of snapshots only) and contains everything but things like caches. It is useful when hardware dies or I make some stupid mistake. Much less time spent on it - more brute force. Some cheap destination is enough for me here.