Ok to Ctrl+C restic backup to Backblaze B2?

Getting Help
1

As I experiment with different b2.connections values to get optimal throughput when using restic 0.12.0 to backup to Backblaze B2, I wonder whether there is any risk whatsoever in aborting a backup with Ctrl+C? Subsequently, I used restic rebuild-index to correct errors found by restic check – is this correct/advised procedure, or could something go wrong when rebuilding the indices?

I am aware that when aborting backups there will be some unreferenced pack files, and that those are no problem for restic. But I wonder: Could such abortion of restic leave behind any incomplete or partial files on the remote repo, which restic could see as corrupted – say, eg, a partial pack file without header? And, could this be worsened by rebuild-index, check or others?

2

Restic does writing on correct order, so (afaik) all you will see is “leftover files” in such case. Next prune would get rid of those anyway.

3

To B2, no, this is not possible. B2 is “object storage” which requires a full and complete upload before any data is reflected in the bucket (same with S3). This is in contrast to other storage protocols like SFTP where it is possible for a file to be partially written. With object storage like S3 and B2, the upload either successfully completes in its entirety, or no file is created. (Except with “large file” or “multipart” uploads, there can be an in-progress multipart upload record, but this doesn’t appear like a file until it is completed. Restic does not need these anyway since the packs are not large enough to warrant using multipart uploading.)

And even if this did happen, prune can determine when a pack file was partially written and will automatically delete them, provided the rest of the rest of the repository looks undamaged.

4

Thanks for the replies! So should I have run restic prune rather than restic rebuild-index to clean up after such Ctrl+C-aborted restic backup then?

5

Yes. rebuild-index is going to include the data in the unused packs in the index, so that backups can deduplicate against it. (This is the mechanism by which backups can be “resumed” – the client will dedupe data it’s already uploaded, even if that data is not referenced by any snapshot yet.)

6

Thanks for your patience! So do I have to presume my repo is corrupt now and I need to restart from scratch?

I do not understand what could have gone wrong with running restic rebuild-index instead of restic prune?

7

It’s hard to say without the complete command history (with output) that led to this repository state.

8

Hm, that’s a bit disheartening. Seems the repository’s integrity is rather brittle then? Any way to reliably check the integrity of the repository in such case (ideally without downloading all data)?

9

No, there just isn’t sufficient information in this thread to determine what the issue is, if there even is one. I don’t think your original post even includes the supposed error message from restic check. If the output was “pack xyz not referenced in any index,” that’s not an error.

10

Ah I see. So the messages I got from restic check were of this form:

pack XYZ: not referenced in any index

Then I ran restic rebuild-index and got messages like this:

adding pack file to index XYZ

And subsequently something like

reading pack files
[0:02] 100.00%  7 / 7 packs
rebuilding index
[0:31] 100.00%  6998 / 6998 packs processed
deleting obsolete index files
[0:02] 100.00%  9 / 9 files deleted
done

After which restic check had no more complaints.

And then I went back and forth between restic backup, Ctrl+C, restic rebuild-index multiple times (as I tuned b2.connections), each time with only output

adding pack file to index XZY

by restic rebuild-index, but not running any further restic check.

Thanks for your help!

11

Right, so that’s more of a notice/warning and doesn’t indicate that there is actually any problem. A pack file not referenced in an index is just evidence of an interrupted backup and causes no harm at all. A future prune would clean up the extra pack if it never becomes referenced by a snapshot. rebuild-index is not necessary in this case.

12

Awesome, thank you so much!