Should rebuild-index check if rest-server is in append-only mode?

Nitpicking, found this today when rebuilding an index to optimize it.

When using Restic with rest-server in --append-only mode, it’ll spend some time trying to figure out how to optimize the index then it’ll write a bunch of errors and end with a backtrace, something like:

Remove(<index/3276140d7a>) returned error, retrying after 298.484759ms: blob not removed, server response: 403 Forbidden (403)
Remove(<index/7af0c8fb38>) returned error, retrying after 400.45593ms: blob not removed, server response: 403 Forbidden (403)
Remove(<index/bf7024ea4f>) returned error, retrying after 507.606314ms: blob not removed, server response: 403 Forbidden (403)
[0:48] 0.00%  0 / 384 files deleted
blob not removed, server response: 403 Forbidden (403)*Backend).Remove

This can take quite a while (specially with restic < 0.12) for large repos, and presumably out of restic’s control when not using rest-server, so should restic check if it’s using rest-server in append-only mode and fail fast with a friendlier error message when trying to rebuild the index? Maybe adding an HTTP endpoint to rest-server that restic can check to figure out if it’s in append-only mode without having to try a delete?

Congrats on Restic 0.12, brilliant release. Thank you.

1 Like

Presumably this will affect other restic commands that delete data from the repo. I never use those though, so not really sure.

Yes, this affects forget, prune and rebuild-index. forget is not that critical, as it “only” tries to delete unneeded snapshot files, but prune and rebuild-index` first write new files and at the last step try to remove the then-obsolete files.

This proceeding however is done for a very good reason: If one of the commands is cancelled for some reason, it should still leave the repo in a sane state. That wouldn’t be the case if first files were removed and then the new files were written.

The problem is - as you correctly pointed out - that restic is not able to detect that deletion is permitted until it attempts to really perform it. With rest-server, the API could be extended, but then this should be also added to other backends which can also implement an append-only repo (e.g. S3). So this would require a major rework in the backend interface.

I agree however that restic shouldn’t retry if a 403 - Forbidden is returned by the backend…

Makes sense. On a second thought, it’d be weird for Restic to behave differently here depending on the backend used.

Thanks for the clarity @alexweiss.