Unable to rewrite snapshot ID xxxx: cannot encode "/" without loosing information

Getting Help
1

Hi, I recently found that the download folder slipt into some of my backups, and I wanted to rewrite them to not have it anymore, though the restic rewrite fails with the message in the subject.

The ID printed is not reproducible across tries and changes:

$ RESTIC_COMPRESSION=max restic rewrite --exclude='Downloads/*'
repository 46211214 opened (version 2, compression level max)

snapshot 0a877585 of [/home/nmelzer] at 2023-05-20 01:00:13.880221453 +0200 CEST)
could not load snapshots: context canceled
Fatal: unable to rewrite snapshot ID "0a877585": cannot encode tree at "/" without loosing information
2 
--exclude='/path/to/Downloads'
3

Tried that, and it seems as if that doesn’t work either:

$ RESTIC_COMPRESSION=max restic rewrite --exclude='/home/nmelzer/Downloads/*'
repository 46211214 opened (version 2, compression level max)

snapshot 08a5e216 of [/home/demo] at 2022-01-25 22:15:54.804224483 +0100 CET)
could not load snapshots: context canceled
Fatal: unable to rewrite snapshot ID "08a5e216": cannot encode tree at "/home/demo" without loosing information
$ RESTIC_COMPRESSION=max restic rewrite --exclude='/home/nmelzer/Downloads/*' --host mimas
repository 46211214 opened (version 2, compression level max)

snapshot 0a877585 of [/home/nmelzer] at 2023-05-20 01:00:13.880221453 +0200 CEST)
could not load snapshots: context canceled
Fatal: unable to rewrite snapshot ID "0a877585": cannot encode tree at "/" without loosing information
$ RESTIC_COMPRESSION=max restic rewrite --exclude='/home/nmelzer/Downloads' --host mimas
repository 46211214 opened (version 2, compression level max)

snapshot 0a877585 of [/home/nmelzer] at 2023-05-20 01:00:13.880221453 +0200 CEST)
could not load snapshots: context canceled
Fatal: unable to rewrite snapshot ID "0a877585": cannot encode tree at "/" without loosing information

And why isn’t in the first run the snapshot not just skipped, as it clearly does not contain any paths to rewrite…

Also, isn’t “loosing information” kind of what we expect from a rewrite?

Or might this be related to that fact that backups are actually created using rustic rather than restic and there is some metadata that restic can not properly move?

4

I did quick test.

restic rewrite --exclude=/Users/kptsky/Downloads/bikuben/

repository 2c57440d opened (version 2, compression level auto)

snapshot c5043d2e of [/Users/kptsky/Downloads] at 2023-05-20 15:26:44.415364 +0100 BST)
excluding /Users/kptsky/Downloads/bikuben
saved new snapshot f35f14a9

snapshot 8d93fc9b of [/Users/kptsky/Downloads] at 2023-05-20 15:26:44.415364 +0100 BST)
excluding /Users/kptsky/Downloads/bikuben
saved new snapshot baf89db9

And bikuben folder with all its content has been removed. I have confirmed it with restic ls

Why it does not work for you I am not sure.

I would check backup consistency.

5

Other explanation is that you use:

--exclude='/home/nmelzer/Downloads/*'

* might be a problem - if there are any links in your Downloads folder leading to content outside…

6

I tried with and without the asterisk.

I am currently checking (found that another machine of mine held a stale lock for 3508h25m34.147066328s :smiley: ).

This check will now take a while. After 5 minutes I’m it is just below 3% of the snapshots.

7

The check reported the repo being fine, just having the unreferenced packs left by rustics lockless purges.

Rewrite still not working.

8

Add some test directory to your repo - then check if you can rewrite it. If no then there is repo problem, if yes then it would point to some “issue” in Downloads folder.

9

We might need some restic elders help here:)

10

I tried random different paths that I would not mind loosing.

I tried against latest and old snapshots, the error remains.

11

I’m running into the same issue. :frowning:

The snapshots on which the rewrite fail for me seems to be a snapshots that was copied from one repository to another.

snapshot a33a1f45 of [/home/user] at 2023-04-25 17:33:42.700107197 +0200 CEST)
could not load snapshots: context canceled
Fatal: unable to rewrite snapshot ID "a33a1f45": cannot encode tree at "/" without loosing information

restic -r sftp:someRemoteRepo snapshots a33a1f45 --json
enter password for repository:
[{"time":"2023-04-25T17:33:42.700107197+02:00","tree":"984af441a9e5de8de4eb27ab9b9a39c3fe38d57c597a9aad42585d4d3f304d18","paths":["/home/user"],"hostname":"<anotherHost>","original":"1c93a52c6a8359c6ccc3b58dd1ee11d60cd1e46f4b1b69ae90f5f9198954eafb","id":"a33a1f4591ea5864398b17fad185cf77d290eca18ef717a1f7ab47e80d8d9f81","short_id":"a33a1f45"}]

edit:

Even running the command restic -r sftp:someRemoteRepo rewrite --dry-run --exclude-file /tmp/exclude a33a1f45 with an empty excludeFile fails

12

That is likely the cause. rewrite verifies that saving the unmodified tree metadata would exactly recreate the existing metadata. That ensures that there are no unexpected changes to a snapshot during a rewrite. In restic 0.16.0 (or using a beta build) restic repair snapshots will be able to fix the problem.

13

@wnklmnn What is the output of restic cat blob 984af441a9e5de8de4eb27ab9b9a39c3fe38d57c597a9aad42585d4d3f304d18? The problems with rewriting the snapshot show up while processing that (tree) blob.