Getting rid of nil subtrees

Getting Help
1

So after a sequence of commands, my repo still has errors in it and I’m trying to clean them up.

This comes in two parts. The first part was run with restic 0.11.0 compiled with go1.15.6 on darwin/amd64 (installed via MacPorts) and is detailed in this bug report, including invoking: rebuild-index, repair, prune.

The second part is with restic 0.12.0 compiled with go1.15.7 on darwin/amd64 (installed via Homebrew). check --read-data or check still shows errors with nil subtrees:

[...]
error for tree 3a95267a:
  tree 3a95267a: dir node "EvernoteBackup.repaired" has no subtree
  tree 3a95267a: dir node "aws-iot-chat-example.repaired" has no subtree
  tree 3a95267a: dir node "base16-iterm2.repaired" has no subtree
  tree 3a95267a: dir node "coursera-calendar.repaired" has no subtree
  tree 3a95267a: dir node "dircolors-solarized.repaired" has no subtree
  tree 3a95267a: dir node "evernote-backup.repaired" has no subtree
  tree 3a95267a: dir node "iTerm2-Color-Schemes.repaired" has no subtree
  tree 3a95267a: dir node "maildir2mbox.repaired" has no subtree
error for tree 01448caa:napshots
  tree 01448caa: dir node "Development.repaired" has no subtree
  tree 01448caa: dir node "Learn.repaired" has no subtree
error for tree b76b1dd3:
  tree b76b1dd3: dir node ".zfunc.repaired" has no subtree
error for tree 8a35f283:
  tree 8a35f283: dir node "hooks.repaired" has no subtree
error for tree 0273712a:
  tree 0273712a: dir node "26.repaired" has no subtree
  tree 0273712a: dir node "69.repaired" has no subtree
  tree 0273712a: dir node "97.repaired" has no subtree
[...]

So I tried another rebuild-index, which looks like it was successful:

# /usr/local/bin/restic rebuild-index
repository 12e94841 opened successfully, password is correct
loading indexes...
getting pack files to read...
rebuilding index
[0:22] 100.00%  173310 / 173310 packs processed
deleting obsolete index files
[0:00] 100.00%  99 / 99 files deleted
done

But check continues to show a bunch of “nil” subtrees.

And, if I run find on one of the broken trees identified in the earlier check --read-data, I get errors:

# /usr/local/bin/restic find f8d2d012
repository 12e94841 opened successfully, password is correct
subtree for node Library.repaired in tree /Library.repaired is nil
github.com/restic/restic/internal/walker.walk
	github.com/restic/restic/internal/walker/walker.go:95
github.com/restic/restic/internal/walker.Walk
	github.com/restic/restic/internal/walker/walker.go:51
main.(*Finder).findInSnapshot
	github.com/restic/restic/cmd/restic/cmd_find.go:267
main.runFind
	github.com/restic/restic/cmd/restic/cmd_find.go:579
main.glob..func10
	github.com/restic/restic/cmd/restic/cmd_find.go:39
github.com/spf13/cobra.(*Command).execute
	github.com/spf13/cobra@v0.0.5/command.go:826
github.com/spf13/cobra.(*Command).ExecuteC
	github.com/spf13/cobra@v0.0.5/command.go:914
github.com/spf13/cobra.(*Command).Execute
	github.com/spf13/cobra@v0.0.5/command.go:864
main.main
	github.com/restic/restic/cmd/restic/main.go:98
runtime.main
	runtime/proc.go:204
runtime.goexit
	runtime/asm_amd64.s:1374

I imagined that rebuild-index would clear out all such errors. Is there no such guarantee?

What can I do now?

2

Sorry for your trouble - seems this nil subtrees are caused by an erroneous functionality in my repair command :unamused: . I’ll change this and add a functionality that lets repair also repair nil subtrees.

Hope you can wait a bit for the correction - I think I can find some time this afternoon (in Germany) to update the PR.

1 Like
3

My PR

now no longer generates nil subtrees and is also capable to repair nil subtrees (if present for whatever reason)

4

@alexweiss That worked and cleared up all my errors.
Thanks for the help and the new command!

I don’t know what the state of the repaired data is, as I don’t yet need to restore any files, but at least I can continue doing backups without interruption and will have these repairs as a backup to the backups.

5

Good to hear! Thanks for the feedback :+1:

Make sure to run a restic check. If you run without --read-data it is also very fast. If this gives no error, your metadata should be completely fine and at least some cheap consistency checks are also done for the data files. (Of course your .repaired files or directories do miss some content)

6

I did restic check --read-data. All good!