--exclude behaving unexpectedly

Triggered by Feature request: Verify · Issue #1280 · restic/restic · GitHub, I decided to verify my backup today.
I backup my /home directory using the following command:

restic backup --exclude /home/michael/.cache /home/michael

I generated a file list of what’s in the repo (via fuse) and diffed that against the contents of my home directory. I noticed that a couple of files were missing:

/home/michael/git/udisks/modules/bcache/data/Makefile.am
/home/michael/git/udisks/modules/bcache/data/org.freedesktop.UDisks2.bcache.policy.in
/home/michael/git/udisks/modules/bcache/data/org.freedesktop.UDisks2.bcache.xml
/home/michael/git/udisks/modules/bcache/Makefile.am
/home/michael/git/udisks/modules/bcache/udisksbcachemoduleiface.c
/home/michael/git/udisks/modules/bcache/udisksbcachestate.c
/home/michael/git/udisks/modules/bcache/udisksbcachestate.h
/home/michael/git/udisks/modules/bcache/udisksbcachetypes.h
/home/michael/git/udisks/modules/bcache/udisksbcacheutil.c
/home/michael/git/udisks/modules/bcache/udisksbcacheutil.h
/home/michael/git/udisks/modules/bcache/udiskslinuxblockbcache.c
/home/michael/git/udisks/modules/bcache/udiskslinuxblockbcache.h
/home/michael/git/udisks/modules/bcache/udiskslinuxmanagerbcache.c
/home/michael/git/udisks/modules/bcache/udiskslinuxmanagerbcache.h

Is this a bug in --exclude or am I misunderstanding how it is supposed to be used.
Can the man page please be clarified how the matching is done.

My question is why /home/michael/git/udisks/modules/bcache/ was apparently matched by /home/michael/.cache

Hm, that exclude pattern clearly should not match the directory. I’m unable to reproduce the behavior you’re seeing, and the pattern (for me) does not match the directory that’s missing from the backup (for me it’s include):

$ tree -a /home/user
/home/user
├── .cache
│   └── foo
└── git
    └── udisks
        └── modules
            └── bcache
                └── data
                    └── Makefile.am

$ restic backup --exclude /home/user/.cache /home/user
scanned 6 directories, 1 files in 0:00
[0:00] 100.00%  0B/s  12B / 12B  7 / 7 items  0 errors  ETA 0:00
duration: 0:00, 0.00MiB/s      
snapshot 89c288c9 saved             

$ restic ls latest
snapshot 89c288c9 of [/home/user] at 2017-09-27 16:23:52.916436947 +0200 CEST):
/user
/user/git
/user/git/udisks
/user/git/udisks/modules
/user/git/udisks/modules/bcache
/user/git/udisks/modules/bcache/data
/user/git/udisks/modules/bcache/data/Makefile.am

Is it maybe possible that the bcache directory was created after the backup was made?

Btw, you can also use restic ls latest to just list the contents of a particular snapshot (or the latest one, with the keyword latest).

Here’s an explanation for the exclude patterns (that still needs to be documented): https://github.com/restic/restic/issues/1005#issuecomment-307204245

the bcache directory has

$ LANG=C stat /home/michael/git/udisks/modules/bcache/
File: /home/michael/git/udisks/modules/bcache/
Size: 4096 Blocks: 8 IO Block: 4096 directory
Device: 816h/2070d Inode: 398495 Links: 3
Access: (0755/drwxr-xr-x) Uid: ( 1000/ michael) Gid: ( 1000/ michael)
Access: 2017-09-27 10:47:12.076281198 +0200
Modify: 2017-07-14 14:19:15.647460437 +0200
Change: 2017-07-14 14:19:15.647460437 +0200

The latest backup is from today.

Fwiw, besides the already mentioned files, the following were also missing:

/home/michael/debian/udisks2/modules/bcache/data/Makefile.am
/home/michael/debian/udisks2/modules/bcache/data/Makefile.in
/home/michael/debian/udisks2/modules/bcache/data/org.freedesktop.UDisks2.bcache.policy.in
/home/michael/debian/udisks2/modules/bcache/data/org.freedesktop.UDisks2.bcache.xml
/home/michael/debian/udisks2/modules/bcache/Makefile.am
/home/michael/debian/udisks2/modules/bcache/Makefile.in
/home/michael/debian/udisks2/modules/bcache/udisksbcachemoduleiface.c
/home/michael/debian/udisks2/modules/bcache/udisksbcachestate.c
/home/michael/debian/udisks2/modules/bcache/udisksbcachestate.h
/home/michael/debian/udisks2/modules/bcache/udisksbcachetypes.h
/home/michael/debian/udisks2/modules/bcache/udisksbcacheutil.c
/home/michael/debian/udisks2/modules/bcache/udisksbcacheutil.h
/home/michael/debian/udisks2/modules/bcache/udiskslinuxblockbcache.c
/home/michael/debian/udisks2/modules/bcache/udiskslinuxblockbcache.h
/home/michael/debian/udisks2/modules/bcache/udiskslinuxmanagerbcache.c
/home/michael/debian/udisks2/modules/bcache/udiskslinuxmanagerbcache.h
/home/michael/git/phpfastcache/bin/stubs/xcache/stub.php
/home/michael/git/phpfastcache-perf/vendor/phpfastcache/phpfastcache/bin/stubs/xcache/stub.php
/home/michael/git/phpfastcache-perf/vendor/phpfastcache/phpfastcache/src/phpFastCache/Drivers/Xcache/Driver.php
/home/michael/git/phpfastcache-perf/vendor/phpfastcache/phpfastcache/src/phpFastCache/Drivers/Xcache/Item.php
/home/michael/git/phpfastcache/src/phpFastCache/Drivers/Xcache/Driver.php
/home/michael/git/phpfastcache/src/phpFastCache/Drivers/Xcache/Item.php

It’s the same pattern though. The repository was originally created with 0.7.1, atm I’m using a git snapshot

Does that happen again when you run restic backup again? Can you please create a debug log and have a look?

Can you try to build a minimal example that I can reproduce?

Since apparently this is not expected behaviour, should I file a bug report? Or do you prefer if we keep it here in the forum?

Oh crap, I guess it was my mistake.
The script which compiled the diff between the backup and my home directory was buggy.

Sorry for the noise :-/

Don’t worry, I’m glad it’s resolved :slight_smile: