Hello,
I completely messed up both my filesystem (containing both the backups and some of the backed up data), but also the mirrored backups.
What happened is that before the disaster occurred, I interrupted the transfer mid-sync (rclone sync --delete-before
). I would have resumed it at a later time, but oops. So the remote repos have most data from previous syncs, but they’re missing any new files + any files that were deleted when creating the latest snapshots (because of --delete-before
).
What I’m trying to do now is recover as much data as possible. I downloaded all the repositories from my mirrors to a local filesystem, and now my idea is that I can restore the last good snapshot for each repository and then try to copy as much data as possible from the following bad repositories.
The problem is that most restic commands will do this:
Load(<data/b7106512f7>, 0, 0) returned error, retrying after 11.222385136s: open repo/data/b7/b7106512f7e80c2b66c273d8de890fe1087f15fb12c8e7762b08cf4b5920fcd9: no such file or directory
Load(<data/c01717c427>, 0, 0) returned error, retrying after 1.807980427s: open repo/data/c0/c01717c4277e85b1f48feb3dbdeba81569cdfc591851e408941936a325c5d308: no such file or directory
Load(<data/c01717c427>, 0, 0) returned error, retrying after 3.343745266s: open repo/data/c0/c01717c4277e85b1f48feb3dbdeba81569cdfc591851e408941936a325c5d308: no such file or directory
Load(<data/c01717c427>, 0, 0) returned error, retrying after 4.659096946s: open repo/data/c0/c01717c4277e85b1f48feb3dbdeba81569cdfc591851e408941936a325c5d308: no such file or directory
There’s no need to retry. The files are lost, forever, they’re not gonna show up if restic asks enough times.
For most of my repos, I can just mount them, an old snap with cp -a
and if I see it’s retrying, try with a smaller set of data. However, that’s way too inconvenient.
I’ve ran rebuild-index
on a copy of the repository and now I’m running check
on it. If that works I’ll update this post, hoping it might be helpful to somebody who experiences the same issue.
But in the meantime I’d like to see if anybody has any suggestions on good ways to handle this kind of situation.