Review snapshots with restic check command

Getting Help
1

Hello everyone,

My name is Simeon and I am quite new to restic.

I have a few questions to ask, and would be extremely grateful for your responses :slight_smile:

When creating snapshots of the pvc’s in our k8s environment, sometimes the backup completes successfully, and sometimes it fails on a random pvc on a random node with the following:

Warning: at least one source file could not be read

I checked the files which were unsuccessfully read and it seems they are just some temporary files that probably get deleted while the process is in progress. That seems OK as those files are not really needed.

But my first question is, if the status of the backup is PartiallyFailed - is it usable and complete (except for the files that were not read)

Secondly, I decided to run restic check on our repo and it returned:

check all packs
check snapshots, trees and blobs
[1:03] 100.00%  212 / 212 snapshots
no errors were found

And when i ran restic check --read-data-subset=1% it returned multiple:

Load(<data/20da56a158>, 24033129, 0) returned error, retrying after 468.466252ms: V4 authentication signed header not found: range
pack <*****> failed to download: StreamPack: V4 authentication signed header not found: range
[2:56] 100.00%  13 / 13 packs
Fatal: repository contains errors

That being said, I am kind of confused, are the snapshots really usable and what might be the issue behind the error thrown above?

Thanks in advance!

2

restic returns exit code 3 in this case to signal that the error is not fatal. The snapshot is intact, it just won’t contain the files about which the backup run complained.

Which restic version and which S3 storage are you using? The failed request definitely includes the range header.

This is the first time I’ve ever seen this error (and Google too). As there are quite a few restic users of the S3 backend, I suspect some weird behavior of the storage backend, when restic tries to read the file.

3

Just for documentation purposes:

I’ve recently seen the exact same authentication error while using some other software that uses the AWS S3 SDK to access an S3 bucket. It was apparently caused by an intermediate HTTPS proxy that for unknown reasons decided to just strip the range header from the request :face_vomiting: .

1 Like