Gdrive 403 rate limit exceeded

When i try to backup on my google drive, mostly i getting these errors: rclone: 2021/05/08 16:57:47 ERROR : locks/660032244f60c6cec488ecf70a8ca7b0fbd46445a9cd5b93debd8b7e943eade6: Post request put error: googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded

Why? Sometimes i dont get these after few retries.
Is there a workaround?

So i can forget restic for backups?

If your storage backend for the repository limits the requests you make you should perhaps consider using another storage. Why use something that limits you, for one of the most important pieces of your infrastructure (your backups).

I have no idea what plan you are on with Google, I would presume that if you are a paying customer you are eligible for a reasonable amount of requests and should not hit a limit like this. But I don’t use Google services and don’t know their pricing.

That said, you could try using the --limit-upload option to the backup command in restic (there’s also a --limit-download option), to see if that helps. It would presumably effectively limit the number of requests per time period.

Tons of people use restic for backups, so I would say “no” to your last question.

There is no other storage that have unlimited space, for just 10 euros :frowning:
BTW there is no problem with rclone alone, only with restic. Few days ago i’ve just backed up 370 gbytes to the same storage with rclone sync.

That’s a low price for as much storage as you’d like, yeah. As long as it’s actually accessible :wink:

It looks like rclone actively rate limits with Google Drive, so that is probably why you haven’t had the problem there. Restic doesn’t do this, it expects that it is free to perform :slight_smile:

What you could do is try to use rclone as the backend for restic, that might actually solve the problem. You’d configure rclone as usual, then reference those remotes as per the previously linked documentation. Also see here for the rclone documentation about it.

today i’ve tried with rclone serve restic, with restic, and the backup works. There are 500 errors (no rate limit, just plain 500 errors), with retries, probably exceeding transfer/sec limit of google which is don’t know how much…

But it works. I did not used any extra parameters for rclone serve, just --buffer-size to make rclone use more RAM for caching.

I don’t know why it doesn’t work if restic directly invokes rclone.