Failed to load snapshot bffe75a2: invalid character '\x02' looking for beginning of value

Hi,
I am facing an issue with restic.

restic version
restic 0.16.0 compiled with go1.20.6 on linux/amd64

I got 2 repos on a google drive.
Everything worked perfectly for months, until I got this error message when I run my usual backup command:

sudo restic --repository-file /etc/restic/home --password-file /etc/restic/secret/home backup --verbose --one-file-system --exclude-caches --iexclude-file /home/me/Restic/excludes.txt /home/me --compression max

Error loading snapshot ******: failed to load snapshot ******:: invalid character ‘\x02’ looking for beginning of value

Could you please help me to fix it?

Sounds like the snapshot file could be corrupted. Can you please see if you get the same when you do restic cat snapshot <the snapshot ID>? You can also try the same with other/earlier snapshots to see if it’s isolated to one or a few snapshots.

Thanks for your message.

restic cat snapshot <the snapshot ID> generates the same error message.

And if I run restic snapshots I get this:

repository 3e1a4a35 opened (version 1)
Ignoring "bd5ecd4694f84ef4f620f4f1751d80e278cb6603c663ce93b7302742caa11d23": failed to load snapshot bd5ecd46: invalid character '\x02' looking for beginning of value
Ignoring "fa584a972d4b00d0ce6b153aa3bba2b892eda0bf9dd12b27749c87b20db54013": failed to load snapshot fa584a97: invalid character '\x02' looking for beginning of value
Ignoring "bffe75a2a6e048ca0a2ed6f02cbd6f8637ccce891fbd92b62a2672e9508b143b": failed to load snapshot bffe75a2: invalid character '\x02' looking for beginning of value
Ignoring "7c5036160a4024c781c9cab7754b0619dce5f75ec56153fb2453e60f59493aa6": failed to load snapshot 7c503616: invalid character '\x02' looking for beginning of value
ID        Time                 Host        Tags        Paths
-------------------------------------------------------------------------
6424b96b  2021-02-28 13:33:15  me                 /home/meDocuments
a5f16353  2021-02-28 16:18:38  me                 /home/meDocuments
acfbc22f  2021-03-01 18:22:24  me                 /home/me
2a4c4587  2021-03-01 22:52:08  me                 /home/me
c695b7e1  2021-03-02 12:02:38  me                 /home/me
[…]
91734b4e  2023-07-14 11:08:10  me                 /home/me
-------------------------------------------------------------------------
163 snapshots

So some of the snapshot files are not what they should be, it seems.

Can you download those files (the ignored ones) from the snapshots/ folder in your repository, and then run sha256sum on them and investigate whether their SHA256 sum corresponds to their filename?

If it does, then the files’ haven’t changed since they were produced/written by restic. I believe that would suggest that there might be a hardware problem, either when running the backup, or when parsing/processing the snapshot files. If so you might want to run memory tests or similar.

It would be interesting to know what data restic does see in those snapshot files.

Anyway, @MichaelEischer probably has some more targeted ideas than I do :slight_smile:

That error is most likely cause by memory corruption (usually CPU or RAM problems) at the host that created the snapshot. The fact that restic complains about an invalid character, means that the snapshot was not modified between creating and retrieving it. Thus, checking the sha256 sum will show that the sha256 hash of the file content and the filename match.

What is the output of restic cat snapshot 7c5036160a4024c781c9cab7754b0619dce5f75ec56153fb2453e60f59493aa6 (or one of the other ignored snapshots)?

Recovering the repository will at least require the following steps:

  • Find out where the data corruption happens: please run a CPU and memory stress test list, prime95, memtest86+ or similar. Also make sure that your system is up to date.
  • Please take a look at the modification timestamps of the ignored snapshot files. Were those created recently?
  • Create a backup copy of the ignore snapshots, then delete the files from the snapshots/ folder.
  • Run restic check --read-data (the will have to download the whole repository) to get an accurate assessment of whether further parts of the repository are damaged.

Thanks for your answer and sorry for delay, but I cannot reproduce the list of the ignored snapshot files.

restic --repository-file /etc/restic/home --password-file /etc/restic/secret/home snapshots
repository 3e1a4a35 opened (repository version 1) successfully, password is correct
could not load snapshot bffe75a2: invalid character '\x02' looking for beginning of value
could not load snapshot fa584a97: invalid character '\x02' looking for beginning of value
could not load snapshot bd5ecd46: invalid character '\x02' looking for beginning of value
could not load snapshot 7c503616: invalid character '\x02' looking for beginning of value
ID        Time                 Host        Tags        Paths
-------------------------------------------------------------------------
6424b96b  2021-02-28 13:33:15  me                 /home/me/Documents
a5f16353  2021-02-28 16:18:38  me                 /home/me/Documents
acfbc22f  2021-03-01 18:22:24  me                 /home/me
2a4c4587  2021-03-01 22:52:08  me                 /home/me
c695b7e1  2021-03-02 12:02:38  me                 /home/me
55a43943  2021-03-02 12:29:25  me                 /home/me
23f860cf  2021-03-02 12:53:57  me                 /home/me
4a84f362  2021-03-02 20:18:07  me                 /home/me
25c49ae7  2021-03-04 19:19:16  me                 /home/me                                                                                                                                                                                                                          
b3d2807b  2021-03-05 15:02:40  me                 /home/me                                                                                                                                                                                                                          
9eb28a45  2021-03-06 00:55:46  me                 /home/me                                                                                                                                                                                                                          
819186ca  2021-03-07 00:53:28  me                 /home/me                                                                                                                                                                                                                          
03990257  2021-03-10 20:43:30  me                 /home/me                                                                                                                                                                                                                          
c19c8b55  2021-03-10 23:05:57  me                 /home/me                                                                                                                                                                                                                          
53e300da  2021-03-12 12:56:26  me                 /home/me                                                                                                                                                                                                                          
746e858c  2021-03-13 21:51:57  me                 /home/me                                                                                                                                                                                                                          
1aa30906  2021-03-16 17:37:21  me                 /home/me                                                                                                                                                                                                                          
3bedb1d3  2021-03-18 17:54:27  me                 /home/me                                                                                                                                                                                                                          
ee4c4ebe  2021-03-20 00:01:21  me                 /home/me                                                                                                                                                                                                                          
559984e3  2021-03-21 00:28:02  me                 /home/me                                                                                                                                                                                                                          
7fc7163f  2021-03-22 16:31:33  me                 /home/me                                                                                                                                                                                                                          
9ceb7ac6  2021-03-23 18:14:02  me                 /home/me                                                                                                                                                                                                                          
2ba245dd  2021-03-24 21:47:54  me                 /home/me                                                                                                                                                                                                                          
c91dcbdd  2021-03-26 01:27:30  me                 /home/me                                                                                                                                                                                                                          
468ae0dc  2021-03-27 13:16:40  me                 /home/me                                                                                                                                                                                                                          
c805b00a  2021-03-27 14:21:56  me                 /home/me                                                                                                                                                                                                                          
06ca1cc1  2021-03-27 18:58:51  me                 /home/me                                                                                                                                                                                                                          
ec7df9e9  2021-03-29 20:02:09  me                 /home/me                                                                                                                                                                                                                          
aafc204a  2021-03-30 23:46:02  me                 /home/me                                                                                                                                                                                                                          
60a868f4  2021-04-01 21:46:00  me                 /home/me                                                                                                                                                                                                                          
c7debce6  2021-04-01 22:37:23  me                 /home/me                                                                                                                                                                                                                          
f0d5e505  2021-04-03 15:34:31  me                 /home/me                                                                                                                                                                                                                          
64f2958c  2021-04-08 13:26:21  me                 /home/me                                                                                                                                                                                                                          
6c775e94  2021-04-13 20:13:47  me                 /home/me                                                                                                                                                                                                                          
e917a684  2021-04-14 15:57:39  me                 /home/me                                                                                                                                                                                                                          
c10b957c  2021-04-14 18:20:30  me                 /home/me                                                                                                                                                                                                                          
79beee36  2021-04-16 18:08:43  me                 /home/me                                                                                                                                                                                                                          
a8661176  2021-04-16 22:43:46  me                 /home/me                                                                                                                                                                                                                          
f663d8bc  2021-04-19 10:59:28  me                 /home/me                                                                                                                                                                                                                          
4b884d55  2021-04-20 17:54:27  me                 /home/me                                                                                                                                                                                                                          
7777be76  2021-04-23 14:29:36  me                 /home/me                                                                                                                                                                                                                          
12020a79  2021-04-26 23:01:52  me                 /home/me
6c94fc71  2021-04-27 12:54:22  me                 /home/me
fe191a6a  2021-04-28 21:19:05  me                 /home/me
089efc66  2021-04-30 00:45:14  me                 /home/me
bf98f6a7  2021-05-02 01:04:32  me                 /home/me
ebc1a492  2021-05-06 20:39:56  me                 /home/me
b8d1eb62  2021-05-08 00:35:07  me                 /home/me
5bd71ae2  2021-05-08 14:45:10  me                 /home/me
4ff7708c  2021-05-14 15:16:13  me                 /home/me
44b93a32  2021-05-18 00:08:52  me                 /home/me
9c07f3d0  2021-05-21 12:54:08  me                 /home/me
d582ddce  2021-05-28 13:07:26  me                 /home/me
c70f824f  2021-06-02 13:14:43  me                 /home/me
5d99210c  2021-06-05 12:33:51  me                 /home/me
f11317c8  2021-06-08 17:53:16  me                 /home/me
07ffc7fc  2021-06-10 17:09:16  me                 /home/me
f48fe8e4  2021-06-10 20:34:48  me                 /home/me
7de63262  2021-06-11 21:42:38  me                 /home/me
1f1159a6  2021-06-12 00:47:50  me                 /home/me
0f30b28e  2021-06-12 01:02:29  me                 /home/me
f73b588a  2021-06-16 00:11:05  me                 /home/me
00f193b1  2021-06-22 12:00:00  me                 /home/me
95e4e644  2021-06-23 22:42:46  me                 /home/me
c1bdfc53  2021-07-02 17:51:46  me                 /home/me
f39ccbb9  2021-07-09 13:14:07  me                 /home/me
5257b71d  2021-07-13 18:52:42  me                 /home/me
f36a4b4a  2021-07-14 00:10:24  me                 /home/me
f18de569  2021-07-18 14:40:03  me                 /home/me
7f0d6164  2021-07-22 12:51:18  me                 /home/me
1e588993  2021-07-27 12:48:20  me                 /home/me
e65c66fa  2021-07-28 18:14:27  me                 /home/me
ee51db0a  2021-07-30 11:37:09  me                 /home/me
ce4640d4  2021-07-30 11:55:23  me                 /home/me
f69e6bae  2021-08-02 12:19:15  me                 /home/me
75a6f866  2021-08-05 12:15:02  me                 /home/me
4eb80f62  2021-08-17 00:40:42  me                 /home/me
d1aebd1b  2021-08-18 12:36:11  me                 /home/me
e7f7019a  2021-08-25 01:15:50  me                 /home/me
567d6a17  2021-08-28 18:59:00  me                 /home/me
faa746a3  2021-09-01 12:37:34  me                 /home/me
ae80eac9  2021-09-05 16:04:59  me                 /home/me
18386a72  2021-09-05 19:35:22  me                 /home/me
fb64572b  2021-09-07 12:13:25  me                 /home/me
e91230f6  2021-09-10 13:02:48  me                 /home/me
a861af33  2021-09-16 16:54:15  me                 /home/me
6cc90bbe  2021-09-20 13:43:13  me                 /home/me
63dafcc5  2021-10-02 17:58:38  me                 /home/me
55ba1003  2021-10-15 19:16:09  me                 /home/me
e6916c90  2021-10-25 16:02:18  me                 /home/me
b7c6a546  2021-11-03 16:15:49  me                 /home/me
a992cab5  2021-11-11 15:23:50  me                 /home/me
956fbc76  2021-11-24 19:09:06  me                 /home/me
ecd7c66d  2021-11-30 01:14:16  me                 /home/me
403d15a4  2021-11-30 19:15:33  me                 /home/me
17adba14  2021-12-02 14:55:51  me                 /home/me
4369cf9e  2021-12-07 01:54:21  me                 /home/me
e2d3fc89  2021-12-15 00:43:16  me                 /home/me
38b08dbe  2021-12-15 12:12:23  me                 /home/me
b205cf7a  2021-12-26 12:43:18  me                 /home/me
be74a833  2022-01-04 01:06:14  me                 /home/me
8f9b4272  2022-01-09 12:40:27  me                 /home/me
c693de44  2022-01-18 17:05:58  me                 /home/me
86cb1fa4  2022-01-21 01:40:02  me                 /home/me
40013edd  2022-02-01 23:25:30  me                 /home/me
86548bd2  2022-02-07 19:22:58  me                 /home/me
d812c1de  2022-02-21 11:04:45  me                 /home/me
4f039883  2022-02-24 00:56:55  me                 /home/me
e428bc4b  2022-03-01 00:23:44  me                 /home/me
1f1a92af  2022-03-05 15:13:16  me                 /home/me
63ef2b37  2022-03-12 17:41:23  me                 /home/me
38a62cb0  2022-03-23 11:23:06  me                 /home/me
00ea5b6f  2022-03-26 01:27:52  me                 /home/me
7e6a027b  2022-03-30 13:45:49  me                 /home/me
278e92c4  2022-04-08 20:15:24  me                 /home/me
20e6a192  2022-04-13 19:25:43  me                 /home/me
29d0901e  2022-04-15 19:38:27  me                 /home/me
a5c12702  2022-04-20 19:16:52  me                 /home/me
fb890b98  2022-04-21 12:20:24  me                 /home/me
90f94411  2022-05-04 12:17:00  me                 /home/me
7c8e8fb1  2022-05-09 15:38:19  me                 /home/me
b8f6edf5  2022-05-19 10:11:31  me                 /home/me
11f5d3f4  2022-05-19 13:02:34  me                 /home/me
5e0f7ddf  2022-05-22 18:48:07  me                 /home/me
098e5295  2022-05-27 20:12:19  me                 /home/me
703ec312  2022-06-07 17:28:23  me                 /home/me
84c3ec21  2022-06-11 01:38:45  me                 /home/me
a6cb2010  2022-06-14 22:40:59  me                 /home/me
4ddb6969  2022-06-20 12:14:46  me                 /home/me
2739c106  2022-06-27 01:13:01  me                 /home/me
3aebedab  2022-07-04 19:38:14  me                 /home/me
c48d5d29  2022-07-07 23:07:33  me                 /home/me
54560fc4  2022-08-01 22:01:32  me                 /home/me
8472b071  2022-08-05 18:44:42  me                 /home/me
faa6b685  2022-08-09 18:46:16  me                 /home/me
f493288b  2022-08-12 11:27:20  me                 /home/me
3772cf6b  2022-08-31 11:38:41  me                 /home/me
98071c8a  2022-09-14 17:13:24  me                 /home/me
13137ab2  2022-09-27 13:28:53  me                 /home/me
1fe187f9  2022-10-29 21:58:10  me                 /home/me
192e9801  2022-11-08 15:53:39  me                 /home/me
de090d53  2022-11-28 15:02:53  me                 /home/me
5af49169  2022-12-12 11:00:55  me                 /home/me
c4dbe94a  2022-12-16 11:52:39  me                 /home/me
cba72870  2022-12-24 00:58:54  me                 /home/me
dcb64a2c  2023-01-01 18:05:10  me                 /home/me
f8e1296e  2023-01-11 14:41:10  me                 /home/me
0fc68fdd  2023-02-07 18:31:46  me                 /home/me
bdf1be80  2023-02-20 11:16:49  me                 /home/me
87b64e8e  2023-02-28 11:40:10  me                 /home/me
ec54589a  2023-03-01 12:06:54  me                 /home/me
91dc46df  2023-03-03 23:17:41  me                 /home/me
9f2f2a7b  2023-03-09 11:18:49  me                 /home/me
dac0de14  2023-04-08 21:03:14  me                 /home/me
dcdcc1c9  2023-04-13 16:04:27  me                 /home/me
ffa1cffa  2023-04-15 23:16:56  me                 /home/me
637d708d  2023-04-28 14:44:18  me                 /home/me
bd0e1fd7  2023-05-18 12:46:14  me                 /home/me
0f46cd09  2023-05-22 17:59:34  me                 /home/me
d3b827da  2023-05-24 14:01:20  me                 /home/me
c54c65fc  2023-06-01 16:14:45  me                 /home/me
ee61a324  2023-06-26 17:36:43  me                 /home/me
91734b4e  2023-07-14 11:08:10  me                 /home/me
-------------------------------------------------------------------------

And if I do restic cat snapshot 7c5036160a4024c781c9cab7754b0619dce5f75ec56153fb2453e60f59493aa6 as suggested, I get the same error message:

repository 3e1a4a35 opened (repository version 1) successfully, password is correct
invalid character '\x02' looking for beginning of value

Regarding the hardware issue, I wonder if it wouldn’t be a no free space issue on my local disk where the restic cache is.

Could you please tell me what do you suggest?

The invalid character errors cannot be caused by running out of free space.

These snapshots are the same as reported by check above. Please check the timestamps of the files in the snapshots/ that start with the prefix about which restic has complained. Did these happen at one point in time or are the spread out over a longer duration?

Hmm, I forgot that there’s no easy way to access broken snapshots.