I’m having significant problems trying to copy large snapshots between a S3 cloud service and a local repo. I’m using restic 0.16.4 on Ubuntu.
Load(<data/af2cd12f5d>, 16867901, 0) returned error, retrying after 919.631755ms: Get "https://s3.somecloudservice.com/bucket/data/af/af2cd12f5dbd16d5a074854df4d61f9d7db398df8f97fa3f0e7db2bbb05e71cc": Connection closed by foreign host https://s3.somecloudservice.com/bucket/data/af/af2cd12f5dbd16d5a074854df4d61f9d7db398df8f97fa3f0e7db2bbb05e71cc. Retry again.
Load(<data/9fbfcd04f0>, 19270402, 0) returned error, retrying after 519.79983ms: Get "https://s3.somecloudservice.com/bucket/data/9f/9fbfcd04f050c9aff31c76f92b8792dc5cffc90290efcda41d4bb7bae3901694": Connection closed by foreign host https://s3.somecloudservice.com/bucket/data/9f/9fbfcd04f050c9aff31c76f92b8792dc5cffc90290efcda41d4bb7bae3901694. Retry again.
Load(<data/9fbfcd04f0>, 19270402, 0) returned error, retrying after 914.866669ms: Get "https://s3.somecloudservice.com/bucket/data/9f/9fbfcd04f050c9aff31c76f92b8792dc5cffc90290efcda41d4bb7bae3901694": Connection closed by foreign host https://s3.somecloudservice.com/bucket/data/9f/9fbfcd04f050c9aff31c76f92b8792dc5cffc90290efcda41d4bb7bae3901694. Retry again.
Load(<data/9fbfcd04f0>, 19270402, 0) returned error, retrying after 1.380435756s: Get "https://s3.somecloudservice.com/bucket/data/9f/9fbfcd04f050c9aff31c76f92b8792dc5cffc90290efcda41d4bb7bae3901694": Connection closed by foreign host https://s3.somecloudservice.com/bucket/data/9f/9fbfcd04f050c9aff31c76f92b8792dc5cffc90290efcda41d4bb7bae3901694. Retry again.
Load(<data/9fbfcd04f0>, 19270402, 0) returned error, retrying after 1.318798524s: Get "https://s3.somecloudservice.com/bucket/data/9f/9fbfcd04f050c9aff31c76f92b8792dc5cffc90290efcda41d4bb7bae3901694": Connection closed by foreign host https://s3.somecloudservice.com/bucket/data/9f/9fbfcd04f050c9aff31c76f92b8792dc5cffc90290efcda41d4bb7bae3901694. Retry again.
Load(<data/9fbfcd04f0>, 19270402, 0) returned error, retrying after 1.786073249s: Get "https://s3.somecloudservice.com/bucket/data/9f/9fbfcd04f050c9aff31c76f92b8792dc5cffc90290efcda41d4bb7bae3901694": Connection closed by foreign host https://s3.somecloudservice.com/bucket/data/9f/9fbfcd04f050c9aff31c76f92b8792dc5cffc90290efcda41d4bb7bae3901694. Retry again.
Load(<data/9fbfcd04f0>, 19270402, 0) returned error, retrying after 3.838571678s: Get "https://s3.somecloudservice.com/bucket/data/9f/9fbfcd04f050c9aff31c76f92b8792dc5cffc90290efcda41d4bb7bae3901694": Connection closed by foreign host https://s3.somecloudservice.com/bucket/data/9f/9fbfcd04f050c9aff31c76f92b8792dc5cffc90290efcda41d4bb7bae3901694. Retry again.
Load(<data/9fbfcd04f0>, 19270402, 0) returned error, retrying after 5.318800934s: Get "https://s3.somecloudservice.com/bucket/data/9f/9fbfcd04f050c9aff31c76f92b8792dc5cffc90290efcda41d4bb7bae3901694": Connection closed by foreign host https://s3.somecloudservice.com/bucket/data/9f/9fbfcd04f050c9aff31c76f92b8792dc5cffc90290efcda41d4bb7bae3901694. Retry again.
Load(<data/9fbfcd04f0>, 19270402, 0) returned error, retrying after 4.354976446s: Get "https://s3.somecloudservice.com/bucket/data/9f/9fbfcd04f050c9aff31c76f92b8792dc5cffc90290efcda41d4bb7bae3901694": Connection closed by foreign host https://s3.somecloudservice.com/bucket/data/9f/9fbfcd04f050c9aff31c76f92b8792dc5cffc90290efcda41d4bb7bae3901694. Retry again.
Load(<data/9fbfcd04f0>, 19270402, 0) returned error, retrying after 10.458154634s: Get "https://s3.somecloudservice.com/bucket/data/9f/9fbfcd04f050c9aff31c76f92b8792dc5cffc90290efcda41d4bb7bae3901694": Connection closed by foreign host https://s3.somecloudservice.com/bucket/data/9f/9fbfcd04f050c9aff31c76f92b8792dc5cffc90290efcda41d4bb7bae3901694. Retry again.
Load(<data/9fbfcd04f0>, 19270402, 0) returned error, retrying after 11.76452645s: Get "https://s3.somecloudservice.com/bucket/data/9f/9fbfcd04f050c9aff31c76f92b8792dc5cffc90290efcda41d4bb7bae3901694": Connection closed by foreign host https://s3.somecloudservice.com/bucket/data/9f/9fbfcd04f050c9aff31c76f92b8792dc5cffc90290efcda41d4bb7bae3901694. Retry again.
[9:51:36] 91.03% 90895 / 99853 packs copied
could not load snapshots: context canceled
Fatal: StreamPack: Get "https://s3.somecloudservice.com/bucket/data/9f/9fbfcd04f050c9aff31c76f92b8792dc5cffc90290efcda41d4bb7bae3901694": Connection closed by foreign host https://s3.somecloudservice.com/bucket/data/9f/9fbfcd04f050c9aff31c76f92b8792dc5cffc90290efcda41d4bb7bae3901694. Retry again.
@MichaelEischer is this related to Issue #4627 ?
Or am I experiencing a different problem entirely? The error messages seem to occur at random when using “restic copy”. Sometimes they happen early in the transfer…other times much later. At this time I’m considering using something like rclone to copy the entire repo on the S3 service and transfer it to a local repo system. Would that be recommended in a situation like this?
Thanks.