I run an sftp backup of /home/$USER over wifi every hour on my Linux laptop, and often when I have multiple screens plugged in and a few windows open, the backup is actually enough to overheat it and shut it down.
I’m trying to figure out why my CPU isn’t throttling enough, but meanwhile is there a way I can slow down the backup? I know of --limit-upload, but I imagine it’s the disk scan that’s computationally intensive, and I don’t think limiting the upload speed would throttle that. Any ideas on how to slow restic down?
I’m wondering: is there so much data that needs to be read by restic on each invocation? Does so much data change between runs? Maybe restic is run in a way so that it re-reads all data for each invocation, that’d explain the CPU load. Usually, I’d expect that subsequent (“incremental”) backups only read the data that has changed.
You can find out if restic works right by looking for the line using parent snapshot [...] when running a subsequent backup (explanation here).
In order to dig into this further:
Can you maybe paste the whole invocation (including the paths you save)?
Is the set of paths you save stable? Or do the paths change on each invocation?
Is the hostname always the same? (check by running restic snapshots)
On Linux there’s ionice which can throttle the IO bandwidth used by restic, but that’ll only change the priority of the IO relative to other processes. Other than that you could try to set the environment variable GOMAXPROCS=1, which should restrict restic to roughly one CPU core at most.
I am always saving /home/kyle. The hostname in this repository is not always the same because I have multiple computers backing up to a single repository:
Which restic version do you use? The output looks like 0.8.3 or older. Please upgrade to restic 0.10.0 or at least 0.9.6. restic 0.9.0 includes a rewrite of the archiver code, so there’s no point in debugging older versions.
Oh! I had no idea I was that far behind. I’ll see what that does. Will there be any backward compatibility issues moving from restic 0.8.3 to 0.10.0, and moving rest-server from 0.9.7 to 0.10.0? I have other computers backing up through the REST server to the same repository.
The repository format itself is compatible, although tracking of file changes works differently which requires restic to re-read all data again. restic 0.10.0 now exits with status code 3 when a warning occured during the backup.
Ok, after updating to 0.10.0 and running it once to allow it to re-read all the data, here’s the output:
Files: 627 new, 738 changed, 480570 unmodified
Dirs: 5 new, 100 changed, 96271 unmodified
Added to the repo: 136.428 MiB
processed 481935 files, 26.097 GiB in 1:37
snapshot 8d71dac8 saved
The restic invocation is the same as before, with the additional prefix GOMAXPROCS=2. With htop I can see restic is running 5 processes. I imagine that’s an implementation detail of the Go runtime.
Half a million files in 97 seconds look like reasonable performance to me. Do you still have the problem with overheating?
Go internally has a lock to ensure that at most GOMAXPROCS threads are running at the same time. When one thread is blocked in a syscall then another one is started to continue processing. So you’ll end up with 2+ threads.
I do still have overheating, but my laptop is also supporting 3 monitors with integrated graphics, so there’s a lot of load on the CPU and the cooling isn’t great. I bet I just need to beef up the exclude file so the whole thing is over sooner.
Processing of excludes is currently not overly efficient, see for example https://github.com/restic/restic/issues/2694 . A few dozen excludes should be ok, everything beyond that will probably slow down your backup. Did you try setting GOMAXPROCS=1 ? With small number of changes this probably doesn’t make much of a difference, but it might be worth a try if you haven’t done so yet.
You could also consider limiting your CPU’s maximum clock speed, either all the time (if overheating is generally a problem) or only while restic is running.