I just started Uploading to Google Team Drives using Restic via the Rclone Backend. I noticed that I get an error every 5minutes when Restic attempts to delete the lock files:
rclone: 2019/01/04 12:13:52 ERROR : locks/0e86719ad377cd9215beec5d2336276ec6a8ca77bee52e232df0f598dd017ddf: Delete request remove error: googleapi: Error 403: The user does not have sufficient permissions for this file., insufficientFilePermissions
Remove(<lock/0e86719ad3>) returned error, retrying after 375.386006ms: blob not removed, server response: 500 Internal Server Error (500)
I tried to delete files manually (using same remote/credentials) using Rclone directly, and I was able to delete the lock files without issue. It seems like Restic doesn’t have the ability to delete files via the RClone Backend. I haven’t set any options for rclone, but it feels like maybe append-only is on somehow? Do I need to turn this off, and how?
I’m not sure if this is an Rclone or Restic issue, but figured I’d start here incase someone already ran into this.
I’ve been using the rclone backend for a while, for Google Drive (started using restic because I wanted to backup to Google Drive) and when I first started using it was via REST server and never had any problems. Once they made the rclone backend I switched and never had any problems. I do know about the Error: 403 and it was harmless in my case. But other than that, I hanven’t had any issues and I do backup, check, forget and prune via rclone backend. This was my “issue”.
What I do also know is that Team Drives are tricky with rclone. Maybe the problem is using restic for a Team Drive. I tried to make a rclone configuration once for the “Computers” section without success. I don’t know how does rclone works with Team Drives but with regular user Drive works.
So the problem only seems to occur with Team Drives. Pruning fails because its unable to delete things too. I did figure out how to work around it, but I’m pretty sure its a bug somewhere (probably rclone)…
Basically Members of the Team Drive need to be a full “Manager” and not just a “Content Manager”.
If “User A” is a “Content Manager” you can use Rclone directly to Delete files as you’d expect. However via Restic (through the Rclone Backend) you get the Permissions Error when it tries to delete. If you change “User A” to being a “Manager” Restic is suddenly able to Delete files.
Interesting… AFAIK, the difference between a Manager and a Content Manager is just that a Manager can manage members of the Team Drive. In theory there should not be any problem with managing files in a Manager user or a Content Manager. If rclone can delete files with the Content Manager permissions, then maybe it is a restic issue.
Good to know that you can store a restic repo in a Team Drive!
EDIT:
Now that I think of this, what I do noticed a while ago about the permissions of a Content Manager is that you can add, edit and “delete” files as a Content Manager but files deleted by a Content Manager cannot be “deleted forever” in the trash can. Only a Manager can delete files forever from a trash can. So, Content Managers can delete files and the system will take care of deleting the file forever in 30 days but they can’t do that manually unless a Manager do it. What I am 99% sure is that a Content Manager permission is not enough to run restic because when restic uses the rclone backend to prune, it uses a rclone option to automatically delete files from trash (if I’m not mistaken is cleanup or something like that); that is why the problem is only with prune (I think). Bottom line, I’m pretty sure that the Content Manager permissions are not enough to use restic with a Team Drive because the only method to backup to a Google Drive is rclone (I think) and the backend will try to delete files from the trash, and because Content Managers can’t delete files from trash it will fail.
Oh, well spotted. I think you might be right. If I’m not mistaken, I think Rclone has an option --drive-use-trash=true/false. I wonder if I can override that to true, it would then allow the delete to happen. I’ll give it a shot and report back.
I suspect that by default --drive-use-trash is true and that is why the same ‘Content Manager’ credentials “appear” to work directly via Rclone when doing a delete… because it actually just going in the trash. For some reason, it must be set to false when called from Restic.
If I’m not mistaken, you can just use rclone to serve restic as a REST server. I don’t think that this option by default forces rclone not to use the trash.
EDIT: right here says how to use rclone serve restic and the --drive-use-trash is available but the default is true. Maybe this approach is useful for you if you don’t want to give the user the “Manager” permissios; that said, I don’t really think is the best option. Worst case scenario, you or someone restore a deleted file from the repo and it may give you errors when running check because it will contain more data than the data referenced in the indexes. Nothing that cannot be fixed by prune if I’m not mistaken but I think it is better to let restic decides what to delete or simply never run prune if you’re afraid of losing data and you have the space available for it.
