Getting a list of excluded files

Getting Help
1

I’m trying to figure out how the --exclude syntax works, and I am struggling a bit. What I’m trying to do is to exclude a folder that is in the root of the working directory, but if a folder with the same name (or file with the same name) is found anywhere else, it should not be excluded. Then there are some more things I’m trying to exclude.

But, my question is about seeing which files are excluded by the exclude syntax. I created a little test working directory and repo to experiment with. What I would like (for this experiment and for my actual setting, is to get a list of all files/folders excluded by the --exclude syntax (or --exclude-file). I tried --verbose=3, but that did not work. Here’s what I was hoping for:

$ export RESTIC_REPOSITORY=repo
$ restic init
[...]
$ mkdir wd
$ touch wd/bla.txt
$ touch wd/ign.txt
[...]
$ restic --exclude=ign.txt --verbose=3 backup wd

Actual output:

[...]
new       /wd/bla.txt
new       /wd/
[...]

Desired output:

[...]
new       /wd/bla.txt
new       /wd/
excluded  /wd/ign.txt
[...]
2

There’s no feature in restic whereby you can get a list of the things that were excluded.

As a temporary workaround you can modify the source code in the cmd/restic/cmd_backup.go file with the following diff, then build restic using go run build.go:

diff --git a/cmd/restic/cmd_backup.go b/cmd/restic/cmd_backup.go
index a6b25834c..0786960ea 100644
--- a/cmd/restic/cmd_backup.go
+++ b/cmd/restic/cmd_backup.go
@@ -610,6 +610,7 @@ func runBackup(opts BackupOptions, gopts GlobalOptions, term *termstatus.Termina
        selectByNameFilter := func(item string) bool {
                for _, reject := range rejectByNameFuncs {
                        if reject(item) {
+                               p.VV("excluded by name: %v\n", item)
                                return false
                        }
                }
@@ -619,6 +620,7 @@ func runBackup(opts BackupOptions, gopts GlobalOptions, term *termstatus.Termina
        selectFilter := func(item string, fi os.FileInfo) bool {
                for _, reject := range rejectFuncs {
                        if reject(item, fi) {
+                               p.VV("excluded by other: %v\n", item)
                                return false
                        }
                }

This will give you output like the following when you use the double verbose option -vv:

$ ./restic -r apa -vv backup foo foo2 --exclude foo2/apa.txt --exclude-larger-than 1k
open repository
repository ba330420 opened successfully, password is correct
lock repository
load index files
using parent snapshot 152bd214
start scan on [foo foo2]
start backup on [foo foo2]
excluded by name: /Users/rawtaz/go/src/github.com/restic/restic/foo2/apa.txt
excluded by other: /Users/rawtaz/go/src/github.com/restic/restic/foo2/peepdf_0.3.zip
scan finished in 0.232s: 3 files, 494 B
unchanged /foo/bar/foo.txt
unchanged /foo/bar/
unchanged /foo/
excluded by name: /Users/rawtaz/go/src/github.com/restic/restic/foo2/apa.txt
unchanged /foo2/blah.txt
excluded by other: /Users/rawtaz/go/src/github.com/restic/restic/foo2/peepdf_0.3.zip
unchanged /foo2/

Files:           0 new,     0 changed,     3 unmodified
Dirs:            0 new,     0 changed,     6 unmodified
Data Blobs:      0 new
Tree Blobs:      0 new
Added to the repo: 0 B  

processed 3 files, 494 B in 0:00
snapshot 265b8dc0 saved
3

Much appreciated! I’m not much of a go builder, so I’ll steer away from doing my actual backups on a home-built binary for now, but this seems like a great way to get started with an actual go project!

If and when this makes it into a release I’ll look forward to using it. In the meantime I guess I’ll just try to be extra careful and specify the exclude strings to be longer and more specific rather than shorter and more general, to minimize the chance of false matches.

4

What are the odds of such a feature being accepted? Or are there other ideas floating around?

5

I can’t imagine we wouldn’t accept such a change, considering that in -vv mode we’re already outputting a line for each path processed. But it should probably be tidied up so there isn’t redundant “excluded” lines in the output, and it would be nice if it’s coherent with the others in terms of that colon. Perhaps putting the “by …” in parentheses instead, if it’s worth keeping them around (personally I think it might be).

6

It’s really super simple to build your own restic, just look here: Installation — restic 0.16.3 documentation

1 Like
7

Yes, it certainly is. It took me all of five minutes to follow these instructions (including installing go, which I hadn’t on my current machine), build restic, and edit the the version command to output a custom message. Wow, this must be the easiest time I’ve had building a custom version of a software package! Thanks again.

1 Like