Process to undo forget & a (clickbait) tale of woe

1

I ran forget against the wrong repo. I wonder if anyone has figured out a process to undo a forget? The blobs are still there, it looks like the index might still be there, so could this info be reversed into recreating a snapshot? OTOH, it looks like any datetime info is lost, so I guess you would have to enter that manually. Could be useful thing to document if it is possible. I don’t know the data structures well enough to do it myself.

The tale of woe is more a cautionary tale. I have several repos: various sub-sets of the filesystem and a testing one. Since security is not a concern for me, I use the same encryption password for convenience sake.

I was working in $HOME and issued a forget. That’s odd, the number of snapshots affected doesn’t make sense with what I know is in the HOME repository. It turns out I had forgotten that last night I was working with the testing repo and had changed the environment variable RESTIC_REPOSITORY to testing. All the changes were happening on testing, the wrong repo! (Luckily, testing and not the other way around .) I had even done a dry-run and not picked up that it was the wrong repo, so that didn’t save me.

After I got over the shock, how to prevent this? The odds of it happening again are high since I have 6-7 repos: say the error rate is 1% with 6 repos pruned weekly, you would affect the wrong repo about 3 times a year, way too often. You could introduce another safety-check into the process by using different passwords for each repo. Although that wouldn’t have saved me since the password was in the RESTIC_PASSWORD environment.

A feature request: environment variables are only used if the -E parameter is used, otherwise all parameters are taken from the command-line. Dunno. Is that a sledgehammer->nut approach? How many people have had this happen to them?

I wonder what checks-and-balances I could introduce into my process to prevent it happening again. In this case, no harm done since it affected testing, but it could easily have been the other way around. I’ve resolved to use a new shell for anything involving production repos to ensure the environment is clean.

2

Thanks for the report, interesting read!

I use -r <repoUrl> when I back up. Hasn’t happened to me :slight_smile:

Even if there was a -E flag, in this case you would have enabled it and the same thing would happen, so that probably wouldn’t have helped.

Not sure there’s a magic bullet fix. If you want to not be prompted, you have to make sure that you’re giving restic the proper arguments, etc. Otherwise you can have a prompt but then it won’t be automated.

Perhaps there should be a default behavior where forget and other commands that “destroy” data asks for confirmation (with some elaborate text so that one might have a chance to identify the mistake before continuing), combined with an argument to skip the prompts, e.g. -y or something? That way you are expected to make sure that you’re giving the proper data to restic before disabling the prompts.

1 Like
3

It is possible, although not implemented. In restic, we have tree and data blobs. Archiving a directory will cause restic to create an new tree object with descriptions of all files in the dir, referencing the data blobs that these files contain. For a sub-directory, the tree of the parent directory contains a reference to the ID of the sub-directory.

A snapshot contains some meta data like the timestamp and references a tree blob. So you could just take the IDs of all tree blobs, then load all tree blobs and remove the IDs from the list which are referenced in a tree blob. Then you’ll end up with the “roots”, tree blobs which aren’t referenced. These are the tree blobs referenced by snapshots. When you then remove all IDs for the still present snapshots, you should end up with the roots of the snapshots you removed.

Please have a look at the design for the details.

Nobody has reported this before.

I’d do the following: replace restic with a shell script for each repo, which is named differently and will unset any environment variable and provide own values for RESTIC_PASSWORD and RESTIC_REPOSITORY. Call them restic-testing and restic-production, for example.

1 Like