Backing up to B2 painfully slow

Despite the fact that my broadband has been upgraded since then, my attempted backup to B2 is much slower than when I backed up to Crashplan a few years ago - at the moment it’s 975.061 KiB/s and I should be able to get at least 1342.77 according to my broadband page.

Are there any tips on improving upload speed? I tried setting connections to 20 in the hopes that might improve things…

Is this an initial backup, or a subsequent one?

Initial. It’s taken almost 24 hours to backup 71GB

Hm, I’m not sure what the limiting factor here is, to be honest. What’s your latency to the B2 service?

How did you try to raise the number of connections exactly?

If you have the space locally, you can also try saving the backup into a local repository and then use rclone to sync it up to B2.

When I did my initial backup it was about 160 GB or so, and it took over night, so definitely not close to 24 hours, for more data than yours. And that was on a computer without AES-NI in the CPU. It was to an SFTP backend.

@fd0 I raised the connections using “-o b2.connections=20” - the full command was:

restic -o b2.connections=20 -r b2:redacted:video /opt/video/redacted/

If I went down the rclone route - would I be able to then use restic to “maintain” those backups on b2? I’ve only really got space for that if I do it piecemeal a directory at a time - which is what I’m trying to do anyway to be honest…

@rawtaz it’s certainly far slower than I would’ve expected. I’d love to use an SFTP backend - if I could find an offsite storage service that was cheap enough…

Finally completed 90GB in 28:43:15…

Is it possible that using the “–force” flag mentioned
here would help?

I suspect it’s worth my trying it anyway - what can you tell m about the implications of using that flag?

Edit: Linked to the wrong place - fixed now.

Currently uploading a 70GB folder and it’s currently at 786.853KiB/s - so it’s not looking any better.

I didn’t include the option to change the number of connections and I did add the force option, but it doesn’t seem to have made a difference.

I’m assuming this is an issue with either my connection or with Backblaze rather than restic - but it’s hard to know how to pin that down…

Hm, interesting case. Can you maybe try using rclone to upload something to B2 and measure the speed? It’s also written in Go, but it uses a different library to access B2.

By the way: how did you measure the upload speed?

I’ll try the rclone idea when this backup has completed (6 hrs to go apparently).

I’m simply taking what restic (or the underlying library) is reporting as the upload speed - is that reliable?

Hard to say for certain since rclone doesn’t report speeds/eta etc., but my initial (completely subjective) opinion is that it’s a fair bit faster. I’ll know more clearly in the morning (in the U.K.).

Edit: Apologies - meant to post this last night. Backup has almost finished - will report soon :slight_smile:

OK - it’s done. Despite positive early signs it rclone only beat a simple restic backup by about an hour and a half - I’m not sure if that’s considered a significant amount?