Exclude with ** doesn't work on fixed path

Getting Help
1

Hi there,

First of all: Thank you very much for the big amount of work everybody spend to make this project happen !

Now to my Problem:
Being on a Linux host, I tried to exclude a lot of files from beeing backuped by using this specific exclude:
“/home/username/Documents/testpath/**/thumpnails”

“~/Documents/testpath/**/thumpnails”
Both of them didn’t work !

I learnd that
“$HOME/Documents/testpath/**/thumpnails”
did the job.

But I would prefer to us a fixed path with a trailing “/home/username” .
I couldn’t figure it out who to do it.
Could somebody plesae point me where to find a solution ?

Greetings

2

I guess that the issue here is that your shell does not expand the tilde character (~) when it is passed to restic in a double-quoted string. So restic gets the literal path ~/Documents/testpath/**/thumpnails (there’s a typo by the way, it’s “thumbnails”, but this is probably unrelated to the issue). In contrast, the environment variable is expanded by the shell in a double-quoted string:

$ echo ~/documents
/home/fd0/documents

$ echo "~/documents"
~/document

$ echo "$HOME/documents"
/home/fd0/documents

This is zsh, I think bash will handle this in the same way. In order to see how restic is started and especially which arguments are expanded, you can prefix your call to restic with ruby -e 'puts ARGV.inspect', like this:

$ ruby -e 'puts ARGV.inspect' restic backup --exclude "~/documents" ~
["restic", "backup", "--exclude", "~/documents", "/home/fd0"]

Here you can see that the shell expanded the ~ but not the "~/documents".

Can you paste your exact command line?

1 Like
3

Yup, it indeed does.

$ echo $SHELL
/bin/bash
$ echo "~/dotfiles"
~/dotfiles
4

I don’t know about you but I have never tried to use the /**/. Instead what I do is $HOME/*/.cache or $HOME/*/.local/share/Trash, for example. I do this and never had any problem. However, looking your test, what I have notice is that in every /*/ you’re telling your terminal to look in one sub-directory. If you add /*/*/ then you’re telling it to look into a second sub-directory, and so on.

$ ls /home/**/*.pdf
ls: cannot access '/home/**/*.pdf': No such file or directory

It’s looking into my user directory and there’s no PDF files in there but there are PDF files in sub-directories.

$ ls /home/*/*/*.pdf
/home/user/Documents/101117.pdf
$ ls /home/*/*/*/*.pdf
/home/user/Documents/Books/advanced-linux-programming.pdf

I think it is better to use a pattern exclude if you want to exclude a lot of different files like PDF files, or PNG files, etc., or to indicate the full path of the directory you want to exclude so it doesn’t get confused and you make sure that what you want to exclude it is actually excluded.

1 Like
5

I think what is happening in your case is that your shell doesn’t have globstar enabled.

moritz@vm:~$ shopt globstar
globstar        off
moritz@vm:~$

This is on a default Ubuntu installation with no changes to how bash behaves.

To turn that functionality on use this command to enable it

shopt -s globstar

If set, the pattern ‘**’ used in a filename expansion context will match all files and zero or more directories and subdirectories. If the pattern is followed by a ‘/’, only directories and subdirectories match.

You can find out more here The Shopt Builtin (Bash Reference Manual)

3 Likes
6

You were totally right. I didn’t knew about that, thank you! I’m using Xubuntu and it is turned off by default.

7

Whoa, thanks a lot for all this answers !!

Yes you are right. My output is:

$ ruby -e ‘puts ARGV.inspect’ restic backup --exclude “~/documents” ~
[“restic”, “backup”, “–exclude”, “~/documents”, “/home/user”]

@moritzdietz
I tried

$ shopt globstar

the output was

globstar off

BUT as i allready wrote in my first post:
to use the exclude:
“$HOME/Documents/testpath/**/thumpnails”
did the job ! So it seems that a least in my Distro (Devuan “ascii”) the globstar switch is somehow circumvented.

I had trouble using a “fixed” path like:
“/home/user/Documents/testpath/**/thumpnails”
… but magically when i tried the option again today everything went smooth & fine !

So i gonna mark my question as solved now.
Thanks a lot for the big bunch of knowledge & help I got !

8

Ah, just one final remark: restic implements ** globbing itself (but only for exclude patterns!), the shell must not expand it.

The issue here was that ~ wasn’t expanded, this is not expanded by restic itself and needs to be handled by the shell.