If I look at my own backups I see an average pack size of 4.7MiB. The largest ones are about 11MiB.
I’d argue that this is too small for any cloud storage system as the round trip times will really be eating into the upload bandwidth. Uploading multiple packs at once will help with this of course. However smaller packs also cost more transactions which cost actual money on some storage platforms (eg s3, b2).
So an option to increase the pack size would
- increase transfer speeds on remote backends (especially on google drive)
- decrease costs (on all backends which charge per transaction)
- produce fewer files (so less likely to breach the 400,000 file limit on team drives for example)
I’m not sure what the downsides would be though. More memory use for definite.
What do you think @fd0 could we make this a parameter to restic?