Did I screw up my repository by backing up to two different b2 buckets from one client?

Getting Help
1

I know there are a million questions and issues logged about >1 client backing up to the same remote repository. But I’ve been unable to find anything about one client backing up to >1 remote repo at the same time (each in their own b2 buckets).

Here’s the story:

I had a backup going that was getting unreasonably large, due to having no exclude rules. So things like VM images and backups, ISO images etc. were in it. (It’s actually a backup of a mirror; the script that creates maintains that mirror is supposed to exclude all that stuff, but obviously not well.)

So I decided to start over with a new bucket. (I’ll delete the old bucket later.)

On top of that, I accidentally killed the first backup job uncleanly. So I figured that I’d check/repair that repo. (Even though it will eventually be redundant and deleted, with fires closing in it seems prudent to not throw anything away just yet and in fact make sure it’s good.)

Here’s where I may have screwed up: While I was still running restic rebuild-index on the first bucket/repository, I also ran restic init and restic backup to the second bucket/repository.

Many hours later, it suddenly occurred to me that restic might only have one cache directory, without awareness of different targets, and both repos might be getting horribly mangled.

So I killed both jobs.

Now I’m running restic rebuild-index on the second bucket/repo. (I’ll worry about the first one later after I figure out what if anything can/should be done.)

When I look in ~/.cache/restic, I do see two subfolders with hex names. I was hopeful that was one for each of the two remote repos. But they both have timestamps later than when I killed the first restic rebuild-index. So, it seems as if the current (second) restic rebuild-index is updating both cache folders? In which case, that would imply there’s only one global cache folder for restic?

So my questions:

  1. Is running two instances at the same time, to two different buckets/repos, indeed not supported?
  2. Did I screw the pooch on one or both repos?
  3. Will a restic rebuild-index, run by itself, fix the second repo? (No errors so far.)
  4. Once that completes, if I run a restic rebuild-index on the first bucket/repo, will that work as well? (Which would imply that either there are either two separate caches, or it gets completely rebuilt with each run of restic rebuild-index?)
  5. And most importantly, what is the meaning of everything?

Much thanks.

2

It is indeed supported. Every repository you restic init gets its own unique ID (which you see the value of in the output from init), and that should be the same as the cache directory holds as you already discovered. So each repository has its own cache directory.

FWIW, I back up my laptop to two different repos at the very same time every day. It works fine.

I can’t see a way that your second repo would be affected by the first one, or vice-versa, assuming you did restic init to create the second one (which you say you did), rather than making a copy of the first one. So no, I don’t think you screwed either of them up.

Is there anything that you feel needs fixing? Are there any symptoms to suggest that?

Unless you did something else than just canceling/killing the backup to the second repository, you really don’t need to do anything with it - it should be fine to just restart the backup and restic will then scan your files again but only upload what hadn’t already been uploaded.

However, since you already started a rebuild-index on the second repository, I would let it run so it finishes. Then start your backup again.

Same thing here, if all you did to the first repository was cancel/kill a running backup job, I don’t see that there’s anything to fix. If you want to keep using it, just re-run your backup to this repository.

FWIW, I often cancel/kill my backup runs, for various reasons. It’s not a problem and I have yet to see it damage a repository.

42

So in summary:

  • Each repository that you individually did restic init to create will get its own ID, and its own cache (in a folder with that ID).

  • You can run backups to multiple repositories at the same time, and you can also do other type of work with restic on multiple repositories at the same time - a restic instance working with one repository will not affect the other repositories or their caches.

  • There might be an exception to the previous point if you were to use the upcoming copy command, but that’s not an issue here in your case at all.

  • Don’t worry, be happy. If you at some point have a reason to think that you screwed up a repository, you can always run restic check to verify its structural integrity, and restic check --read-data to also verify that the actual data stored in the repository hasn’t changed on disk. If these two passes, your repository should be fine.

Hope that helps, let us know if you have follow-up questions!

PS: The latest master build has some serious optimizations for the check command, so next time you start a check, I would recommend using that build instead of a release (until restic v0.10.0 is released).

1 Like
3

Bingo, that answers it! Phew!

Paranoia. I suppose I could just let the first repo be, and only bother with a rebuild 1) if I wind up having to resort to it, and 2) if something is wrong.

Good to know. Social proof.

Thanks, your feedback has been super helpful.

1 Like