Please note that these problems arise when a backup runs for the first time, basically when there’s a lot of new files to deal with.
It might have to do with the fact that I also have a glusterfs mount active that serves the directory that I am backing up. Even though I access the local storage instead of the network mount, I still think it is detecting activity and thus goes to work, although not as bad as accessing the network mount itself. Although I’m not 100% sure this is the case.
The thing is, CPU Usage is quite low during backup, memory usage is not the problem either (I think), it’s the load, there’s just so much to do at once that everything gets queued up and the cpu just can’t keep up and other programs die cause of that extensive load and everything becomes unresponsive…
To my surprise and short linux experience: high load is not the same as high cpu usage. I’m also in an openstack environment where the server is influenced by outside factors (neighbors or other factors) and although this shouldn’t be the case, it is in practice. Software probably behaves more as expected in a non-virtual environment.
To be fair, the backups are running great again now that I lowered the GOGC and I really don’t want to change anything now. I will fall back to nice and ionice when I run into problems again.
I’m using 0.9.0.20 now, but to be honest, the restic version didn’t really matter, there was high load anyway before the gogc setting was changed.