Error when creating a backup via the rest server

Backups through the rest server suddenly stopped working. Everything worked and then it stopped.
When trying to backup, I get multiple errors:

repository 59e91fba opened successfully, password is correct
Load(<index/46d49beb32>, 0, 0) returned error, retrying after 582.280027ms: <index/46d49beb32> does not exist
Load(<index/5502460939>, 0, 0) returned error, retrying after 468.857094ms: <index/5502460939> does not exist
Load(<index/119b02d70c>, 0, 0) returned error, retrying after 462.318748ms: <index/119b02d70c> does not exist
Load(<index/6ce2584c18>, 0, 0) returned error, retrying after 593.411537ms: <index/6ce2584c18> does not exist
Load(<index/5e25a131fb>, 0, 0) returned error, retrying after 282.818509ms: <index/5e25a131fb> does not exist
Load(<index/5e25a131fb>, 0, 0) returned error, retrying after 492.389441ms: <index/5e25a131fb> does not exist
Load(<index/119b02d70c>, 0, 0) returned error, retrying after 447.727139ms: <index/119b02d70c> does not exist
Load(<index/5502460939>, 0, 0) returned error, retrying after 600.683895ms: <index/5502460939> does not exist
Load(<index/46d49beb32>, 0, 0) returned error, retrying after 761.409471ms: <index/46d49beb32> does not exist
Load(<index/6ce2584c18>, 0, 0) returned error, retrying after 985.229971ms: <index/6ce2584c18> does not exist
Load(<index/5e25a131fb>, 0, 0) returned error, retrying after 803.546856ms: <index/5e25a131fb> does not exist

what happened and what to do about it?
The rest server is running in the mode append-only. At the same time, I can get a list of snapshots without problems.

It seems that at the time when the backups stopped working, I used forget and prune with specified tag in repo

Please always include the commands you ran when showing output. If possible, also include the output of that forget and prune you ran.

You should start by looking in your rest-server’s file system to see if you can find those files, to verify that they are indeed not there.

Sorry if I did not explain well. I have one repo where backups are performed from two hosts, host1 and host2. Host1 have tags etc, home, var. Host2 have tags etc, home, dbs

Im used command restic -r repo1/ forget --tag dbs --keep-daily 7 --keep-weekly 5 --keep-monthly 12
and it ended without errors

repository 59e91fba opened successfully, password is correct
Applying Policy: keep 7 daily, 5 weekly, 12 monthly snapshots

after im used command restic -r repo1/ prune

repository 59e91fba opened successfully, password is correct                                                                                                                       [613/1909]
loading indexes...
loading all snapshots...
finding data that is still in use for 164 snapshots
[0:03] 100.00%  164 / 164 snapshots
searching used packs...
collecting packs for deletion and repacking
[0:01] 100.00%  86482 / 86482 packs processed

to repack:           16 blobs / 4.462 MiB
this removes         12 blobs / 7.969 KiB
to delete:        52105 blobs / 75.983 GiB
total prune:      52117 blobs / 75.983 GiB
remaining:       295133 blobs / 353.739 GiB
unused size after prune: 0 B (0.00% of remaining size)

repacking packs
[0:00] 100.00%  4 / 4 packs repacked
rebuilding index
[0:01] 100.00%  71200 / 71200 packs processed
deleting obsolete index files
[0:00] 100.00%  142 / 142 files deleted
removing 15285 old packs
[0:01] 100.00%  15285 / 15285 files deleted
done

after i used command restic -r repo1/ check --read-data

created new cache in /tmp/restic-check-cache-355914110
create exclusive lock for repository
load indexes
check all packs
check snapshots, trees and blobs
[0:04] 100.00%  164 / 164 snapshots
read all data
[3:24:28] 100.00%  71200 / 71200 packs
no errors were found

now from host1 I execute the command restic -r rest:repo1 --tag etc backup /etc/

and I get errors from 1 post

That sounds fine and to be expected. But you said you also ran prune and that this is when the problems started? Do you have the full command and its output from that? Also, did you check the files mentioned in the error message yet, to see if they are there on the rest-server’s disk or not?

1 Like

where can I find them?
I completed the previous post

they are not in the Index folder,Or are they not the full names in the error message? Then it seems they are

if this is important, then I did not execute the forget and prune commands through the rest-server, but directly on the rest-server file system. Also, the versions are different, on host1 0.12, on the rest-server 0.12.1

I’m an idiot. I was running a command on the local file system of the server from a different user and the index files acquired other rights not available to the rest server process. But thanks for the tip to look at the files anyway. Please delete the topic)

3 Likes

Ha! Don’t worry about it, probably everyone and their cat has had wrong permissions due to running something as another user at some point :slight_smile: I’d like to keep the topic so that others can learn from the error message and find solutions, I hope that’s fine with you. There’s absolutely nothing to be ashamed of here :hugs:

2 Likes

Thanks for keeping the topic! :heart_eyes: :wink:
I made the same mistake too. :neutral_face: At least for me it was not that obvious (until I faced this once) that just running an application as super user can even stop this applications to work for other users. Although it seems to be just consequent use of unix permission system.
Thank you guys!

PS: Maybe one can mark thw last post of @rdev as solution?

Good idea, done!