Python script ignoring --exclude

Getting Help
1

hello,

I’m trying to stretch my n00b python skills by writing a script for backing up my /home
on my Desktop Linux box. but for testing purpose, I’m skipping the /home/dthomas folder as it’s huge
and only backing up the other users on the system.

Without embarrassing myself with the code here are the important snippets
I’m using rclone as a rate limiter but I don’t feel this is causing the issue as it’s backing up without any other issues.

#Folders to backups
backupData = "/home"
excludeFolders = ["",'"/home/dthomas"']

excludeData = " --exclude=".join(excludeFolders)

def rRestic():
    logger.info("starting restic backup")
    subprocess.run(["restic", "backup", backupData, excludeData])
    logger.info("finished Rclone backup")

the problem is when I run the script I get the following output

~/Bin >>> python3 PiroResic.py                                                                                                                          [130]
INFO:__main__:starting Rclone
2019/01/30 13:22:00 NOTICE: B2 bucket rclonethadeus: Serving restic REST API on http://127.0.0.1:8080/
INFO:__main__:starting restic backup
 --exclude="/home/dthomas" does not exist, skipping
open repository
repository 28d64023 opened successfully, password is correct

any tips would be awesome.

2

You have a few problems in how you pass arguments. Both apparently stem from a misconception that subprocess.run() is going to parse your argument array like your shell parses a command line, and this is wrong.

  • You must pass one flag per list item and, while you don’t currently do this, your code appears set up to expect this to happen later when you add another item to excludeFolders.
  • The exact string is going to be passed to restic, including the double quotes and leading whitespace. Both of these would be erased by your shell, and restic would not see them. But it does see them, and that causes problems.

Because your flag begins with a space, restic is not parsing it as an argument and thinks that you literally want to back up a directory called ' --exclude="/home/dthomas"' (without the ' characters).

If not for the space, restic would think you want to exclude a directory "/home/dthomas" with the double quotes in its name, and this isn’t correct either.

The string join you’re doing is not only a bit messy, but it doesn’t even work. List comprehensions are your friend; also, you can concatenate lists with +.

backupData = ['/home']
excludeFolders = ['/home/dthomas']
excludeArgs = ["--exclude=%s" % i for i in excludeFolders]

...

subprocess.run(["restic", "backup"] + backupData + excludeArgs)
3

sorry, how embarrassing. :blush: I’m a total python noob.
I thought It would make sense to learn by doing rather than reading. so I’m writing a wrapper for my backup needs.

Thank you for your advice, I will try and learn from it :smiley:
I would never have thought to

excludeArgs = ["--exclude=%s" % i for i in excludeFolders]

thank you again :blush:

4

No need to be embarrassed. Everyone here was a noob at one point.

You might consider writing a shell script instead of a Python script. There’s little extra that Python offers you for this particular task, as the script is likely going to be very simple anyway.