Slow speeds of Restic with Google Drive rclone backend

fd0 · July 20, 2018, 6:28am

No, not easily. And I’m very reluctant to exposing this as a user-configurable option. If you want to experiment with it though, it’s easy to change in the source code here:

github.com

restic/restic/blob/67535e00a86469a0cebebd240f421b15199bbac2/internal/repository/packer_manager.go#L39


      
          }
          
          // packerManager keeps a list of open packs and creates new on demand.
          type packerManager struct {
          	be      Saver
          	key     *crypto.Key
          	pm      sync.Mutex
          	packers []*Packer
          }
          
          const minPackSize = 4 * 1024 * 1024
          
          // newPackerManager returns an new packer manager which writes temporary files
          // to a temporary directory
          func newPackerManager(be Saver, key *crypto.Key) *packerManager {
          	return &packerManager{
          		be:  be,
          		key: key,
          	}
          }

You can try setting minPackSize to something much larger than the default, like 64MiB (or even 256MiB), and try that. I think restic will cope just fine with larger packs, but I haven’t tried it in a long time. So please report back!

I think there’s no one-size-fits-all for packs, there are several trade-offs to decide and for now we go with a small pack size by default. In the long run I’d like to use a dynamic value which increases the pack size when restic detects that the backend connection has a high bandwidth.