Deleted repository by mistake - Clone or Start from scratch

First, I use 3 different repositories to backup my computer:

  1. External SSD: I bring the disk at work as a first offsite backup
  2. rsync.net: An other offsite backup
  3. raspberry pi: I have a local Pi with enough storage to store an other repo

The problem I have is that yesterday I deleted the repo on the Pi4. It cannot be repaired for sure.

I could start with a new repo, but I wonder if I could clone 1 of the repo I have to all snapshots.

Is it safe to clone an existing repo to be used in parallel with the original one ? The repo will never be merged.

Thanks

Edit: Itā€™s 3 restic repositories that were initialized individually (no clone or copy). rsync.net is a SFTP repo accessed using the rclone backend

Not sure Iā€™m entirely following since you havenā€™t even once mentioned that you are actually using restic, while at the same time you do mention rsync. Which of the three repositories you mentioned have you backed up to using restic?

Generally speaking, you can indeed simply copy the files and folders that make up a restic repository, to another place, and keep using that copy of the repository. It will have the same master encryption key and the same password key of course, but you can replace the current/old key with a new one it you want. So yes, you can copy a repository and thereby have two repositories.

You might find that the two repositories have the same ID though, and Iā€™m not sure how that will work with the cache restic uses locally, someone else will have to answer that. Iā€™m pretty sure it wonā€™t be an actual problem though.

Restic will regard both copies to be the same repository which partially ā€œbreaksā€ the cache. As long no two restic runs access both copies in parallel, the cache should work without throwing errors (which wouldnā€™t break your repository though).

However, the cache no longer works properly: when loading a repository, restic removes files which no longer exist in the repository from the cache. With two diverging repositories this will have the effect that over time fewer and fewer data is actually cached, as every file which is not contained in both repositories will be dropped when switching between both copies.

1 Like

They are all restic repositories and they were initialized individually (no clone nor copy). The second repository in my list is a repository hosted on https://rsync.net/ using rclone SFTP backend.

Thatā€™s what I fear. I would have 2 repositories with the same ID and itā€™s unknown what could really happen. To be safe Iā€™ll probably just start a new repository.

Unless there is a way to change the repository ID, it doesnā€™t feel safe to use 2 repositories that have the same ID. It might work now but the real problem might appear later. I would try to change the ID but the config file is encrypted. I was able to change the ID of a borg repo that suffered the same fate on that machine.

edit: fixed typos

Thereā€™s no easy way to change the ID of a repository in restic right now. Iā€™ve hacked together a small command to change the ID: https://github.com/MichaelEischer/restic/tree/change-repo-id . To use it youā€™ll have to checkout that branch, then compile it using go build -tags debug ./cmd/restic and then use the ./restic debug changeID i-understand-that-this-could-break-my-repository-and-i-have-created-a-backup-of-the-config-file command to change the repository id to a random new one. Make sure to run this command on the correct copy of the repository and create a backup of the config file in the repository first.

The only remaining small downside is that both repository copies would still shared the same encryption key. This could be solved by using the copy PR (https://github.com/restic/restic/pull/2606) to copy the snapshots from the old repository to the new one.

3 Likes

Thanks ! Iā€™ll have a look at it later today. Oh, and Iā€™ll be extra careful to work only on the cloned repo. Iā€™ll follow your advice and make a copy of the config file, just in case.

Iā€™ve tweaked the command a bit to make it less likely to accidentally modify a repository. The call now also expected to get the repository id ./restic debug changeID i-understand-that-this-could-break-my-repository-and-i-have-created-a-backup-of-the-config-file repoID. Youā€™ll have to run it one time with a placeholder instead of the real repository id, and then take the real id from the error message:
Fatal: expected repository id repoID, found 3dd0878cc15a49638e8a8cd4a36d004a2dd1342ea64bd01bc0b8a414b2d2711e, aborting.

For that example the complete restic call would be ./restic debug changeID i-understand-that-this-could-break-my-repository-and-i-have-created-a-backup-of-the-config-file 3dd0878cc15a49638e8a8cd4a36d004a2dd1342ea64bd01bc0b8a414b2d2711e.

1 Like

Iā€™ve done the modification on the repository clone and all looks fine.

  • The original repository (id: 88e9a296)
$ restic hdd snapshots -c
Sourcing '/home/morgoth/Restic/repos/hdd/hdd.env' ...
/dev/mapper/luks-b0848365-1952-4acf-9be4-59b837f602b5 /mnt/backup
[INFO] Filesystem '/mnt/backup' is mounted
Repository: /mnt/backup/restic-backup/morgoth
repository 88e9a296 opened successfully, password is correct
  • The cloned repository after changing the ID (id: 2b570354)
$ restic picloud snapshots -c
Sourcing '/home/morgoth/Restic/repos/picloud/picloud.env' ...
Repository: rclone:sftp:/data/backups/restic/morgoth
rclone: 2020/08/22 20:58:40 NOTICE: Config file "/home/morgoth/.config/rclone/rclone.conf" not found - using defaults
repository 2b570354 opened successfully, password is correct
  • When I access the repositories I get 2 different caches. To verify that I deleted the content of ~/.cache/restic and accessed the repository to verify which directory is created.
$ ls -al ~/.cache/restic
total 4
drwx------ 1 morgoth morgoth  280 Aug 22 20:59 .
drwxr-xr-x 1 morgoth morgoth 2756 Aug 22 20:54 ..
drwx------ 1 morgoth morgoth   50 Aug 22 20:58 2b570354032f1143614a5ae79a10f4bd94d1ee6e6fc028bb5078256aee83f7f1
drwx------ 1 morgoth morgoth   50 Aug 22 20:59 88e9a296f59b3df16854cd1f902c8e40b5a6ff316cacdbeb094aac71ff5c8f1a
-rw-r--r-- 1 morgoth morgoth   44 Aug 22 20:54 CACHEDIR.TAG
[21:00:25] morgoth@morgoth:~/.cache/restic
$ 

Iā€™m currently checking the ā€˜newā€™ repo (check --read-data) just to be sure itā€™s in a good ā€œstateā€

So based on these verification I would say that changing the ID on the repo clone was successful.

I would say that this does not matter much for me. Itā€™s to backup my personal data. Itā€™s good enough that itā€™s a repository protected with a password.

Thanks a lot !

Edit: After completing check on the repository I successfully tested restore. I selected random files in random snapshots and I was able to restore without problem.

2 Likes