restic 0.9.1 compiled with go1.10.3 on darwin/amd64
Repo is on Wasabi (s3:https://us-east-1@s3.wasabisys.com)
Timeline :
- restic check
–>1 additional files were found in the repo, which likely contain duplicate data.
You can runrestic prune
to correct this. - restic prune
- restic rebuild-index
- restic check
As I’m on a real slow connection on one of my machines, I must confess I did some Ctrl-C during backup or check (and perhaps prune).
The messages are “tree not found in Index”, do that mean some part of the repo are missing ?
If so, as an end user (even a bit techy), how do I know to which file or snapshot these errors belongs to ?
I’m in the process of trying a --read-data-subset 1/10, but that outputs the same errors, so it’s helpless.
Is the --read-data flag making some fixing ?
Shouldn’t restic tell me to forget the culprit snpashot ? Or perhaps flag the culprit file as bad if the snapshot is to keep ?
I think these kind of messages should be for a debug level, as a normal user I hope for a message like
“file /bla/bla in snapshot xxxx is corrupted, run --read-data to fix it” or “file /bla/bla in snapshot xxxx is corrupted, you should forget snapshot xxx”.
Is --read-data fixing things ? Or is it just a confirmation the real data is different in the repo vs on the source ?
Could it be fixed by re-fetching the corrupted file ? (as far as I understood restic, this would generate a new snapshot, no ?)
Do that mean the whole repo is corrupted ? Or only one or some snapshots ?
$ restic -r s3:https://us-east-1@s3.wasabisys.com/my-backup rebuild-index
repository 2c128c6e opened successfully, password is correct
counting files in repo
pack file cannot be listed 824981796608e9e4d16bd6534e461c304f7e4d876106fa4e56206381be698d0f: EOF
[35:54] 100.00% 132671 / 132671 packs
finding old index files
saved new indexes as [fa145cb3 0bc1df87 4e185daa 3fee5dd5 83d391d4 14614059 2558d313 ed0d127f de82203e f7111dc4 f15c6a1c 6750607b b2cae93a 8635cba1 b86af3cb cda9055e da279e96 6405aa99 08aca003 edce9582 9c9055e3 7447af9c c788189d 01edeb63 10a2f9ea 981cab99 492c24be 7a65bedc 9a395cab 97851de1 0757a523 b50fad91 a29bed50 7495e5d8 58f5af8b e7557c47 5d066599 ab9b2ebe a3dde579 bd455dc3 aa18e796 c4043df0 53c12dbc 0a6b256f a20178ad]
remove 1359 old index files
$ restic -r s3:https://us-east-1@s3.wasabisys.com/my-backup check
using temporary cache in /var/folders/r1/3sr1p9391ql86sbhkyzw8qk80000gn/T/restic-check-cache-920683911
repository 2c128c6e opened successfully, password is correct
create exclusive lock for repository
load indexes
check all packs
pack 82498179: not referenced in any index
1 additional files were found in the repo, which likely contain duplicate data.
You can run `restic prune` to correct this.
check snapshots, trees and blobs
error for tree 1b3c7305:
tree 1b3c73055917aaae934bf5ec8ec2537c3bb7d60358f0661f8fc9830820180096 not found in repository
error for tree 0e02d4e1:
tree 0e02d4e15d50157f88c1491d62c04cb8be5f75274daf15ce91f41f746e0f5fdd not found in repository
error for tree 3657ccc4:
tree 3657ccc484e6b125a8a2ea3f1c03a7ddfb301aceeeae10316b683479706470ba not found in repository
error for tree e90739d7:
tree e90739d7a4eeffd7e4968e0aa6d99802f6217867afe7083279f1acfd71281619 not found in repository
error for tree 183965da:
tree 183965da7985686f59d30a60fd7b0e40ab0d8d0b2a1f93842e0090171d062988 not found in repository
error for tree 7bb16c02:
tree 7bb16c022e926cade02ee53dee1f6c3a4da3eeb6bbc406ca3610adc99e273f46 not found in repository
error for tree 3440509d:
tree 3440509dc76f90b922b6e0a42864a7b3a7d7ad00576d6c10fa62195c083e75f4 not found in repository
error for tree 03a48092:
tree 03a48092c6e9d1225b2a3c19ead844abcb0738e8a893e8ae0f504e353e839b97 not found in repository
error for tree 8753b186:
tree 8753b1863cdcfb4a5d91e2679c4010039c58d024a13da38871558c1da0bcf015 not found in repository
error for tree 1c8fdbb2:
tree 1c8fdbb247b22a4c205c6863b2aeb59a046faa5e13b75bb908c71e46ca2fa63e not found in repository
error for tree 8cb3472f:
tree 8cb3472fd490820980a5f6813bbcee91e078875a9fd316a02a3b29bdd8be3ab6 not found in repository
error for tree ca7447c6:
tree ca7447c62e848a9de6c53737d24145d5598015ae43c9e9d8db97aa71eb4a1469 not found in repository
error for tree ec3a2b55:
tree ec3a2b55fc2fcb291869ce17b35067566a11184af3f82d33432f42850f40667a not found in repository
error for tree ef7e8e0d:
tree ef7e8e0ddb37be06e94330985018089d98f77b2682eeb07270f4457ff7e85174 not found in repository
error for tree a452188b:
tree a452188b3f74111ec793e7812454c7ba6ffe35e9e84cf9ba572b5a2774f07645 not found in repository
error for tree 5e118e8e:
tree 5e118e8e0d3cff06ce185fcde7af8e27c22488edb55f12d76a8464c22093d4b6 not found in repository
Fatal: repository contains errors
$ restic -r s3:https://us-east-1@s3.wasabisys.com/my-backup rebuild-index
repository 2c128c6e opened successfully, password is correct
counting files in repo
pack file cannot be listed 824981796608e9e4d16bd6534e461c304f7e4d876106fa4e56206381be698d0f: EOF
[29:38] 100.00% 132671 / 132671 packs
finding old index files
saved new indexes as [470f61f3 e9d66ca9 c73b8917 cfcf80ac 2acf1e1d ea50b481 b2ffb59b d4508542 242c83a8 9376b69a 945d6b70 d5603c37 6de3add4 18cc8ac4 2683aa09 fa3b9413 042d9f9c 0b52a567 b77d9800 c525e722 378a3fe9 86ec589b 8f6df71f 218385e0 747f7cbc c62d6fb6 32d7312e c862a140 e8b03ba6 c816f9c6 e524c1a7 e295e321 34a68b8e a528288f 6d4bcbec 28cdc531 42dc22ab a608494d ab154703 641b2832 77397e72 f174e0c3 bcd23b39 0a351469 82f1bf0e]
remove 45 old index files
I’m now in the process of trying a --read-data-subset 1/10, and it gives me the exact same list of errors.
$ restic -r s3:https://us-east-1@s3.wasabisys.com/my-backup check --read-data-subset 1/10
using temporary cache in /var/folders/8z/rnjlwcm15pv1rbhq2q34mmr00000gn/T/restic-check-cache-058842757
repository 2c128c6e opened successfully, password is correct
create exclusive lock for repository
load indexes
check all packs
pack 82498179: not referenced in any index
1 additional files were found in the repo, which likely contain duplicate data.
You can run `restic prune` to correct this.
check snapshots, trees and blobs
error for tree 1b3c7305:
tree 1b3c73055917aaae934bf5ec8ec2537c3bb7d60358f0661f8fc9830820180096 not found in repository
error for tree 0e02d4e1:
tree 0e02d4e15d50157f88c1491d62c04cb8be5f75274daf15ce91f41f746e0f5fdd not found in repository
error for tree e90739d7:
tree e90739d7a4eeffd7e4968e0aa6d99802f6217867afe7083279f1acfd71281619 not found in repository
error for tree 3657ccc4:
tree 3657ccc484e6b125a8a2ea3f1c03a7ddfb301aceeeae10316b683479706470ba not found in repository
error for tree 183965da:
tree 183965da7985686f59d30a60fd7b0e40ab0d8d0b2a1f93842e0090171d062988 not found in repository
error for tree 03a48092:
tree 03a48092c6e9d1225b2a3c19ead844abcb0738e8a893e8ae0f504e353e839b97 not found in repository
error for tree 8753b186:
tree 8753b1863cdcfb4a5d91e2679c4010039c58d024a13da38871558c1da0bcf015 not found in repository
error for tree 7bb16c02:
tree 7bb16c022e926cade02ee53dee1f6c3a4da3eeb6bbc406ca3610adc99e273f46 not found in repository
error for tree 3440509d:
tree 3440509dc76f90b922b6e0a42864a7b3a7d7ad00576d6c10fa62195c083e75f4 not found in repository
error for tree ca7447c6:
tree ca7447c62e848a9de6c53737d24145d5598015ae43c9e9d8db97aa71eb4a1469 not found in repository
error for tree ef7e8e0d:
tree ef7e8e0ddb37be06e94330985018089d98f77b2682eeb07270f4457ff7e85174 not found in repository
error for tree 8cb3472f:
tree 8cb3472fd490820980a5f6813bbcee91e078875a9fd316a02a3b29bdd8be3ab6 not found in repository
error for tree ec3a2b55:
tree ec3a2b55fc2fcb291869ce17b35067566a11184af3f82d33432f42850f40667a not found in repository
error for tree 1c8fdbb2:
tree 1c8fdbb247b22a4c205c6863b2aeb59a046faa5e13b75bb908c71e46ca2fa63e not found in repository
error for tree 5e118e8e:
tree 5e118e8e0d3cff06ce185fcde7af8e27c22488edb55f12d76a8464c22093d4b6 not found in repository
error for tree a452188b:
tree a452188b3f74111ec793e7812454c7ba6ffe35e9e84cf9ba572b5a2774f07645 not found in repository
read group #1 of 13437 data packs (out of total 132670 packs in 10 groups)
[3:27:25] 23.73% 3188 / 13437 items
As you can see, 24 % of the job took 3,5 hours.
As I already have the list of errors can I safely stop this process ?