Second backup never ends

Getting Help
1

Hello,

I’ve been using restic since few weeks. On a debian server (with no issues) and on a osx system (10.12.6). In both scenario I backup whole system using rclone backend to Google Drive.

On the osx system after the first backup (216Gi, two or three days to complete) I created a LaunchDaemon in /Library/LaunchDaemons to run a backup every 30 minutes.

/usr/local/bin/restic -o rclone.program=/usr/local/bin/rclone -p /var/root/restic.psw --exclude-file /var/root/rclone-exclude.txt -r rclone:restic:Folder backup /

Anyway the second backup never finished. I did several attemps and after few days I just killed the restic backup process. The current one is running since 9 days and according to nettop it downloaded 11 GiB and uploaded only 1334 MiB, that seems fishy to me.

This is the exclude list:

/private/tmp/**
/tmp/**
/.**
/Volumes/**

Other info:

$ restic version
restic 0.9.1 (v0.9.1-50-g67535e00-dirty) compiled with go1.10.3 on darwin/amd64
$ rclone version
rclone v1.42
- os/arch: darwin/amd64
- go version: go1.10.3
2

That sounds very odd and may be a bug, can you please fill out the issue template over at GitHub so we can find out what’s going on? Thanks!

3

I was going to fill out the issue template at GitHub and I run the backup command from shell instead of executing it from the launch daemon, this is the output:

[17:31] 2.71%  92916 files 6.007 GiB, total 1682676 files 221.743 GiB, 0 errors ETA 10:29:05

so I’m going to wait for those ten hours to finish and check if the process stay appended. if so the issue may be with the launchdaemon

4

So, I removed the existing backup and did a new one from scratch from the cli. It completed in almost 50 hours. Then I did a second backup also from cli completed in 5 hours ca. But when I try to run the backup from the launch daemon it stays appended for days with higher downloaded data than uploaded as reported in the first message.

Here’s my plist for the daemon:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
  <string>some.label</string>

  <key>ProgramArguments</key>
  <array>
    <string>/usr/local/bin/restic</string>
    <string>-o</string>
    <string>rclone.program=/usr/local/bin/rclone</string>
    <string>-p</string>
    <string>/var/root/restic.psw</string>
    <string>--exclude-file</string>
    <string>/var/root/rclone-exclude.txt</string>
    <string>-r</string>
    <string>rclone:restic:Folder</string>
    <string>backup</string>
    <string>/</string>
  </array>

  <key>StartInterval</key>
  <integer>60</integer>

  <key>RunAtLoad</key>
  <false/>
</dict>
</plist>

restic 0.9.1 (v0.9.1-71-g8c124a2b-dirty) compiled with go1.10.3 on darwin/amd64

rclone v1.42
- os/arch: darwin/amd64
- go version: go1.10.3
5

So, I guess you can reproduce this, correct? Does it always hang/download when launched via launchd?

I’d say the next step would be to build restic with the debug tag (go run build.go -tags debug), then start it with the environment variable DEBUG_LOG set to a file name. restic writes an extensive debug log (which will get large very fast) and we should be able to see what’s going on.

First, it’d be helpful if you could try this with a much smaller data set, like a single directory with just a few files. Make a backup, make a second backup, then try this via launchd, and see if it completes.

6

oki, I found the problem is in launchd: it doesn’t inherit environment variable, so no $HOME was set and restic complained about not finding the cache. After adding to the plist the following it’s working as expected:

  <key>EnvironmentVariables</key>
  <dict>
    <key>HOME</key>
    <string>/var/root</string>
  </dict>

Indeed Example: Scheduling regular backups on macOS states:

Use .bashrc or .bash_profile to export any environment variables your restic command may need, for example, the repository password. (This is convenient because it also allows you to get them in any Terminal you start.)

But I thought it was related only to specific restic env variables don’t I didn’t need since I pass everything as command params

I’m sorry I pointed out the problem for restic while it was my setup (and my scarce knowledge of launchd ;))