OK, so first of all, there is an easy solution to this, which is “pay for a non-free Backblaze account”. But I like to do things the difficult way …
So I set up a backup to a Backblaze free 10Gb account a year or so back. I watched it for a few days and it all seemed good, so I let it run. Recently I logged into the Backblaze account and saw that the size of the bucket had been growing inexplicably fast, so I investigated. It turns out that there was a stale lock in the bucket, and so the weekly prune command had not been running. restic forget --prune --verbose --keep-daily 7 --keep-weekly 4 --keep-monthly 6
So I took a look, (restic snapshots)and sure enough, there were a zillion snapshots that hadn’t been removed, so I ran the prune command manually.
After it had been running for 20 mins I started getting errors, and eventually it stopped, as Backblaze was sending back 403 messages. What had happened was that the prune process had triggered the daily bandwidth cap of 1Gb, from which I gathered that the prune process downloads each block, repacks it locally and then re-uploads it.
So, I have to wait for a day before I can try again. But as there are a lot of files to prune, due to it not happening for a year, I’m guessing it will happen again if I run the same command. What would be the best approach to move towards completing the prune process. Is there a way to minimize the amount of data downloaded?
Related question: The prune was rudely interrupted. I’m hoping that I’ll just be able to pick it up again and nothing will be broken … any views on this?
--max-repack-size size if set limits the total size of files to repack. As prune first stores all repacked files and deletes the obsolete files at the end, this option might be handy if you expect many files to be repacked and fear to run low on storage.
Indirectly it will also limit bandwidth used.
Interruptions are harmless. But they leave behind unreferenced packs (they will be cleared next time). If you keep interrupting it again and again you will exhaust your limited space available. If it happens then read how to recover here.
OK, I think that worked. I had to run a backup operation first, so that the prune had ‘something to do’.
Then I used --max-repack-size 500M and it deleted a few of the packs.
The next day I backed up again and ran the same prune command.
I think one more repetition tomorrow and it will sort itself out. Thanks for the pointer.
With respect to available space on Backblaze, that was actually OK, I had enough room on there, but it was the 1Gb Cap on downloads that was hitting me. This was a 900Mb backup, that had ballooned to 5Gb over a year, so that’s what alerted me in the first place, and that’s now down to about 2Gb.
Incidentally, I see a setting in Backblaze for Lifecycle
Keep all versions of the file (default)
Keep only the last version of the file
Keep prior versions for this number of days:
Use custom lifecycle rules:
I’m presuming Keep Only the Last Version is the setting to use, given the nature of incremental backups?
