Scale Rest-Server


I have a server with 2 x 10 G connection, which are combined via a LAGG. The plan was to be able to backup as many devices as possible at the same time. On the server runs the rest server.

However, as soon as I start an inital backup from 50 servers, I have a CPU load of 100% on the backup server and get a maximum throughput of 300 MB/s. - Even with more servers I get a maximum of 300 MB/s.

To exclude the possibility that this is due to limitations of my network setup or the storage, I have already made appropriate tests. Both the hard drives and the network are capable of much more.

Therefore I have the assumption that it is because of the high CPU load.

The specs of the server:
Network: 2 x 10G
CPU: 2 x XEON E5 2620 v4
Disks: 147 x 1.9 TiB in zraid2

I am grateful for any tips to improve the performance.

1 Like

Oh, if you have a large scale scenario, it would be really cool if you could test my Rust implementation of rest-server:

It is still an alpha version and I would not yet suggest using it in a production environment. But I’m planning to finish this small project soon.

1 Like

I’ve actually looked at your project before, I can see if I can convince my boss to let us do some tests with it.

The performance bottleneck could be due to how rest-server checks the credentials (I’m assuming you use .htpasswd for authentication). That is probably fixed by Cache basic auth credentials by MichaelEischer · Pull Request #138 · restic/rest-server · GitHub .


Just wondering. Could https encryption be the limiting factor?

All traffic runs through an internal network, so we have dispensed with tls

Thanks for the tip, I’ll keep that in mind!