I just started Uploading to Google Team Drives using Restic via the Rclone Backend. I noticed that I get an error every 5minutes when Restic attempts to delete the lock files:
rclone: 2019/01/04 12:13:52 ERROR : locks/0e86719ad377cd9215beec5d2336276ec6a8ca77bee52e232df0f598dd017ddf: Delete request remove error: googleapi: Error 403: The user does not have sufficient permissions for this file., insufficientFilePermissions
Remove(<lock/0e86719ad3>) returned error, retrying after 375.386006ms: blob not removed, server response: 500 Internal Server Error (500)
I tried to delete files manually (using same remote/credentials) using Rclone directly, and I was able to delete the lock files without issue. It seems like Restic doesn’t have the ability to delete files via the RClone Backend. I haven’t set any options for rclone, but it feels like maybe
append-only is on somehow? Do I need to turn this off, and how?
I’m not sure if this is an Rclone or Restic issue, but figured I’d start here incase someone already ran into this.
This option does not prohibit deletion of locks.
I’ve been using the
rclone backend for a while, for Google Drive (started using restic because I wanted to backup to Google Drive) and when I first started using it was via REST server and never had any problems. Once they made the
rclone backend I switched and never had any problems. I do know about the
Error: 403 and it was harmless in my case. But other than that, I hanven’t had any issues and I do
rclone backend. This was my “issue”.
What I do also know is that Team Drives are tricky with
rclone. Maybe the problem is using
restic for a Team Drive. I tried to make a
rclone configuration once for the “Computers” section without success. I don’t know how does
rclone works with Team Drives but with regular user Drive works.
Well there goes that thought
After this run finishes, I’ll try it out on a normal Drive and report back.
So the problem only seems to occur with Team Drives. Pruning fails because its unable to delete things too. I did figure out how to work around it, but I’m pretty sure its a bug somewhere (probably rclone)…
Basically Members of the Team Drive need to be a full “Manager” and not just a “Content Manager”.
If “User A” is a “Content Manager” you can use Rclone directly to Delete files as you’d expect. However via Restic (through the Rclone Backend) you get the Permissions Error when it tries to delete. If you change “User A” to being a “Manager” Restic is suddenly able to Delete files.
I’ll raise the question over on the Rclone forum.
Interesting… AFAIK, the difference between a Manager and a Content Manager is just that a Manager can manage members of the Team Drive. In theory there should not be any problem with managing files in a Manager user or a Content Manager. If
rclone can delete files with the Content Manager permissions, then maybe it is a
Good to know that you can store a
restic repo in a Team Drive!
Now that I think of this, what I do noticed a while ago about the permissions of a Content Manager is that you can add, edit and “delete” files as a Content Manager but files deleted by a Content Manager cannot be “deleted forever” in the trash can. Only a Manager can delete files forever from a trash can. So, Content Managers can delete files and the system will take care of deleting the file forever in 30 days but they can’t do that manually unless a Manager do it. What I am 99% sure is that a Content Manager permission is not enough to run
restic because when
restic uses the
rclone backend to
prune, it uses a
rclone option to automatically delete files from trash (if I’m not mistaken is
cleanup or something like that); that is why the problem is only with
prune (I think). Bottom line, I’m pretty sure that the Content Manager permissions are not enough to use
restic with a Team Drive because the only method to backup to a Google Drive is
rclone (I think) and the backend will try to delete files from the trash, and because Content Managers can’t delete files from trash it will fail.
Oh, well spotted. I think you might be right. If I’m not mistaken, I think Rclone has an option
--drive-use-trash=true/false. I wonder if I can override that to
true, it would then allow the delete to happen. I’ll give it a shot and report back.
I suspect that by default
true and that is why the same ‘Content Manager’ credentials “appear” to work directly via Rclone when doing a delete… because it actually just going in the trash. For some reason, it must be set to
false when called from Restic.
If I’m not mistaken, you can just use
rclone to serve
restic as a REST server. I don’t think that this option by default forces
rclone not to use the trash.
EDIT: right here says how to use
rclone serve restic and the
--drive-use-trash is available but the default is
true. Maybe this approach is useful for you if you don’t want to give the user the “Manager” permissios; that said, I don’t really think is the best option. Worst case scenario, you or someone restore a deleted file from the repo and it may give you errors when running
check because it will contain more data than the data referenced in the indexes. Nothing that cannot be fixed by
prune if I’m not mistaken but I think it is better to let
restic decides what to delete or simply never run
prune if you’re afraid of losing data and you have the space available for it.