I have to admit I didn’t read the changelog before upgrading: I started to look into the changes after restic started complaining. I remember doing a rollback, some “mv”, and at the end I discovered the migration procedure, so it could be I messed up fiddling with the directories.
I’m not sure, but I suppose I manually renamed the snapshot and key folder, after that I tried the repo migration and I think the data directory format wasn’t migrated because I already migrated the key directory. Is it possible? After that I didn’t had to read from the data packs and I didn’t noticed the issue until the first restic mount…
Due to some issues I have no longer access to that repo’s copy ( I had to reconstruct it as explained here), but looking into the logs I’ve found a rclone sync after the repo upgrade: I have a bunch of snapshots in the snapshot folder deleted and the same in the snapshots folder uploaded as new. The same for the key file, moved from the key folder to the keys folder.
Nothing changed in the data directory that day, so I just started to have new files in the data/xx folders, while the old files remained in the data folder. I thought it was ok, but I never had the chance to properly test it: less tnat two months after the repo migration I had a drive failure and I discovered the issue while trying some restores.
2024/10/29 23:46:23 INFO : keys/fb9991e76ecce086d23b070340efec89f1fad8f5dea917d27eb833b9624db0e1: Copied (new)
2024/10/29 23:46:23 INFO : snapshots/00425fb261d0e824b9d15b087e100042c5c5e3e712d614dec73e7dc451ca554a: Copied (new)
2024/10/29 23:46:23 INFO : snapshots/00bc41042060d6d1596dd9a62735b1d2f944fd4f02faa133287a735395d390ce: Copied (new)
2024/10/29 23:46:23 INFO : snapshots/01c8e9522f058f90682e284f0f8fa58b0f8aa2c8dca074e5666035f73a5cca44: Copied (new)
2024/10/29 23:46:23 INFO : snapshots/0216bfac231e1d609663699b62b9277aaa8313d28b937be9159c79cea178d41b: Copied (new)
2024/10/29 23:46:23 INFO : snapshots/0332dc8c0e0b3d33e1807b7927edac76d6f9e1a4ed4e654489d3d8a65ee54df4: Copied (new)
2024/10/29 23:46:23 INFO : snapshots/0409a2cde3010d1ed3faac4953a113be756e7ff2aeff1565331c59dfe57fca76: Copied (new)
– output truncated –
2024/10/29 23:53:05 INFO : snapshot/6d1622504f91b4c90d7c0a333610b962cd0bb6e3134723de916acf2193de4a83: Deleted
2024/10/29 23:53:05 INFO : snapshot/e2f71c9cec6df86eb7d7b534b03c40f291a84d4737e0cba3b17079b9629b0865: Deleted
2024/10/29 23:53:05 INFO : snapshot/86f4e932aab0e7515889149bd4524979c9eb6f09f523e0631e787edcc6c1e62d: Deleted
2024/10/29 23:53:05 INFO : key/fb9991e76ecce086d23b070340efec89f1fad8f5dea917d27eb833b9624db0e1: Deleted
2024/10/29 23:53:05 INFO : snapshot/4f22debffbaaaefc49af8a32bfdadf34fc9d7cdd4976bf12ff07700a46c3bcdc: Deleted
2024/10/29 23:53:05 INFO : snapshot/1d42ce1e9ba39751f07edc3e3f12e355d3bf34e611afaf305d0d34db71d8e359: Deleted
2024/10/29 23:53:05 INFO : snapshot/f69981d8981f78f5c95bed41cbdabe8fc3821c0e5affffcda9f4278391799299: Deleted
Lesson learned:
- check the data: I now execute daily a restic check with read-data, just 1/30 of the repo each day.
- read before upgrade: You don’t want to discover a deprecation after the upgrade
- restic is hard to break: Despite my attempts I was able to rebuild a working repo without losing a single bit
- you cannot really have your backup data in AWS: if you really need to recover from a disaster you’ll pay A LOT! I don’t like saying that, but roughly 100$ per Tb just for the data transfer is too much for me.
I also have a suggestion: I would like an error message in restic upgrade before a deprecation, a repo format change, or anything else that you need to address immediately. When ready you could issue a flag or an env variable to let the upgrade proceed.