Worth a shot. Could you also open a bug report issue on GitHub? If you don’t feel like it, let me know and I can do it, it’s just cooler if you do it.
1 Like
The crash will disappear if you pass --quiet. This bug is triggered when the status line exceeds the terminal width. Making sure that the terminal is always 40+ characters wide should also help.
I’ve opened a PR to fix this crash: https://github.com/restic/restic/pull/2899
A single Load([...]) [...] does not exist warning shouldn’t cause problems as restic retries the operation up to ten times. What worries me more is that in your screenshot the same file is listed twice.
1 Like
@MichaelEischer Does this mean there’s no sense in duplicating the issue by opening another bug report?
@rawtaz Hmm, still getting some interesting results with --quiet
pack 86831381: not referenced in any index
pack 94bf0989: not referenced in any index
pack c0e32d88: not referenced in any index
62 additional files were found in the repo, which likely contain duplicate data.
You can runrestic pruneto correct this.
check snapshots, trees and blobs
read all data
rclone: 2020/08/26 09:24:12 ERROR : data/1f/1f8796e28435cd6ab2092c205dd27c0ed85267b8f97990c6ee81039c11d5c56a: Didn’t finish writing GET request (wrote 620576/5138113 bytes): read tcp 10.137.61.27:60859->74.120.8.224:443: read: connection reset by peer
Load(<data/1f8796e284>, 0, 0) returned error, retrying after 720.254544ms: unexpected EOF
Pack ID does not match, want d596975c, got 79cfb256
It’s still going. Does this mean I should try to find d596975c in an earlier version and replace it if the sums don’t match??
Finally got an offline copy… and now I’m getting this. Starting to think my repo is hosed 
check snapshots, trees and blobs
error for tree 8d225dfa:
decrypting blob 8d225dfa7ce38645ef026e5b8314b6475d65c2d7f1f6e6edd9b00778b73ee794 failed: ciphertext verification failed
Hmm, even with the --quiet switch I get this now:
successfully removed locks
incomplete pack file (will be removed): 416036aa0ca7eacae9d45c1dc73f8f3e78c0b36b13e88010d77edd590547b67c
incomplete pack file (will be removed): a661505028d0772716b3fefc602b5ef63bdc692a5c155c0ab57846dde8c8f094
incomplete pack file (will be removed): af6474b3856499e64b7e3db2a19159ad482b1a0aaf7481416a5b543a45534595
incomplete pack file (will be removed): b707cc9fca1b0dc2b13fe4418fa16c3e5eac7b7f0b49f27ec8b9482548bd0f27
incomplete pack file (will be removed): b87e1edaca3363be11908cf60f1bc9dffdde363c0e402802f55c254b58ea39b6
incomplete pack file (will be removed): d596975c62d722ac6859a017839a5ec9679fd4dc5f8b5b4eb7e4d2bc502a87cf
incomplete pack file (will be removed): dbcc1edf0210249cf8be551059719ea97f992c9cbba311f3244e62565b96d877
incomplete pack file (will be removed): e7551bb95e1557fe46e96dd79b8cd5831c9488d938c84786c1874cca1931f38a
incomplete pack file (will be removed): fa169b6b79a62d6fc13e6e7ba1d59ad8d789b7d7ebdaf8d520819a2030433448
id 8d225dfa7ce38645ef026e5b8314b6475d65c2d7f1f6e6edd9b00778b73ee794 not found in repository
github.com/restic/restic/internal/repository.(*Repository).LoadBlob
github.com/restic/restic/internal/repository/repository.go:152
github.com/restic/restic/internal/repository.(*Repository).LoadTree
github.com/restic/restic/internal/repository/repository.go:723
github.com/restic/restic/internal/restic.FindUsedBlobs
github.com/restic/restic/internal/restic/find.go:19
main.getUsedBlobs
github.com/restic/restic/cmd/restic/cmd_prune.go:276
main.pruneRepository
github.com/restic/restic/cmd/restic/cmd_prune.go:158
main.runPrune
github.com/restic/restic/cmd/restic/cmd_prune.go:62
main.glob…func18
github.com/restic/restic/cmd/restic/cmd_prune.go:27
github.com/spf13/cobra.(*Command).execute
github.com/spf13/cobra@v0.0.3/command.go:762
github.com/spf13/cobra.(*Command).ExecuteC
github.com/spf13/cobra@v0.0.3/command.go:852
github.com/spf13/cobra.(*Command).Execute
github.com/spf13/cobra@v0.0.3/command.go:800
main.main
github.com/restic/restic/cmd/restic/main.go:86
runtime.main
runtime/proc.go:203
runtime.goexit
runtime/asm_amd64.s:1373
Okay. I rclone’d over my local repo, and started over. This time instead of rebuild-index and prune both, I just ran rebuild-index. Then check. Now I essentially have two errors:
repository 24344e18 opened successfully, password is correct
created new cache in /var/folders/lj/z4b97q0n1wb64qrrb0rnnrbw0000gn/T/restic-check-cache-546465070
create exclusive lock for repository
load indexes
check all packs
pack d596975c: not referenced in any index
pack 416036aa: not referenced in any index
pack dbcc1edf: not referenced in any index
pack af6474b3: not referenced in any index
pack e7551bb9: not referenced in any index
pack a6615050: not referenced in any index
pack b707cc9f: not referenced in any index
pack b87e1eda: not referenced in any index
pack fa169b6b: not referenced in any index
9 additional files were found in the repo, which likely contain duplicate data.
You can runrestic pruneto correct this.
check snapshots, trees and blobs
error for tree 8d225dfa:
id 8d225dfa7ce38645ef026e5b8314b6475d65c2d7f1f6e6edd9b00778b73ee794 not found in repository
error for tree 786a54f4:
tree 786a54f4: file “2020-07-13 10.23.06.jpg” blob 2 size could not be found
tree 786a54f4: file “2020-07-13 10.23.06.jpg” blob 3 size could not be found
tree 786a54f4: file “2020-07-13 19.08.24.jpg” blob 0 size could not be found
tree 786a54f4: file “2020-07-13 19.08.24.jpg” blob 1 size could not be found
tree 786a54f4, blob dce4b0d9: not found in index
tree 786a54f4, blob f870a0d3: not found in index
tree 786a54f4, blob b8591f64: not found in index
tree 786a54f4, blob a5a9c2a3: not found in index
read all data
[9:16:04] 100.00% 223668 / 223668 packs
Fatal: repository contains errors
akrabu@akrabu-macbook-air ~ % restic -r /Volumes/Archive/Backup/restic-repo/ find --tree
8d225dfa 786a54f4
repository 24344e18 opened successfully, password is correct
Found tree 786a54f469ff29ac043b21b9116f12867cae175e7db84097a219f2597488a5db
… path /C/Users/Herb/Dropbox/Camera Uploads
… in snapshot 4ae87449 (2020-07-27 10:41:49)
Unable to load tree 8d225dfa7ce38645ef026e5b8314b6475d65c2d7f1f6e6edd9b00778b73ee794
… which belongs to snapshot 6afeb332bf6b9c880cf71eab20791b5812750bc46bd06b66a4c79e57f5
5395f4.
restic stats -r /Volumes/Archive/Backup/restic-repo/
akrabu@akrabu-macbook-air ~ % restic stats -r /Volumes/Archive/Backup/restic-repo/
repository 24344e18 opened successfully, password is correct
scanning…
walking tree 8d225dfa7ce38645ef026e5b8314b6475d65c2d7f1f6e6edd9b00778b73ee794: id 8d225df
a7ce38645ef026e5b8314b6475d65c2d7f1f6e6edd9b00778b73ee794 not found in repository
Okay I forgot 6afeb332 and 4ae87449. Let’s see what check --read-data does this time.
Hooray! Working repository. No errors. I rclone’d that up to pCloud. Going to try a local prune and another check. If that works out I think I’m set.
I’m still very curious about the “ciphertext verification failed” earlier, but all that is now gone.
The PR is already merged by now, so no there’s no need to open an issue now.
These messages contradict each other. Did you run both commands on the same repository copy? Did you run rebuild-index in-between?
The switch only helped with the specific crash in the check command.
The first time, I rclone’d my repo from pCloud to a local drive, ran rebuild-index, then prune, then check --read-data - and got the ciphertext error.
I then re-rclone’d over my local copy to reset it from pCloud, and this time tried just rebuild-index, then check. Afterwards I was left with only two bad snapshots. I removed those two snapshots, ran a check --read-data, and was finally left with a working repo - which I then rclone’d back into pCloud.
I think I’m going to keep this local copy, and only run prune jobs locally after rcloning down from the cloud, then only uploading back once check --read-data gives me the all clear. Not sure if it’s my connection or pCloud or both, but it’s just too unstable for prune jobs with restic+rclone. Never had this problem with b2 - so it’s either pCloud (likely) or the way restic+rclone handles things a little differently than just restic alone.
I will say, for long copy jobs, rclone often throws a lot of errors on both b2 and pCloud on my connection. It seems to handle the small sync changes well enough, though. So I can’t completely rule out CenturyLink as the culprit (with restic alone handling the poor connection better than restic+rclone).
So if I see an error like this, does it just mean there was a temporary hiccup, and the retry worked? I wish there was a follow-up message so I know whether or not the retry was successful.
check snapshots, trees and blobs
Load(<snapshot/f4d5ec8bc8>, 0, 0) returned error, retrying after 720.254544ms: <snapshot/f4d5ec8bc8> does not exist
no errors were found
Every failed operation is retried up to 10 times. So if that retry message is only printed a few times, then the operation worked eventually. If the operation still fails after the 10 retries this usually leads to an additional error message.
So the log from that check run looks kind of ok. But it is still odd, that a file that has existed for quite some time can temporarily lead to a “does not exist” error. It means that someone (either pCloud or the API usage in rclone) seriously messed up the error handling(?) or their replication consistency.
1 Like
Hi, what’s your experience with pCloud in the meantime? Currently there are black friday offers again.
I actually have 4TB through pCloud. You can buy it twice and have it applied to the same account.
That said I gave up on using Restic with it. They don’t handle partially uploaded files correctly. It will save them, where b2 or others will discard them. So over time you’ll have silent corruption of your repository. Sucks cause it would be perfect with a lifetime account but yeah don’t trust it.
Thanks for your answer. That’s sad, because their black friday lifetime offers are really a good deal, e.g. at the moment 245 € for 2 TB 
EDIT: BTW: Did you try to reach the pCloud support about that issue?
Well, honestly, there is a major benefit in using restic + pcloud:
If you are using the EU datacenter of pCloud, then all data is hashed using SHA1 and SHA256. Starting with rclone 1.57 you have trivial access to the SHA256 of files stored in pCloud (e.g. using lsjson --hash). So you can easily compare the path name of restic objects (aka filename=SHA265 hash of object) with the stored hash in pCloud, which gives you some kind of “outer” check for the repository.
It’s not as great as an “inner” check like “restic check --read-data”, but it’s super fast and most of the time sufficient.
Hi,
are you using pCloud with restic without problems?