Am I getting my keep* right?

I’m currently trying to set up a script to run forget/prune and I’m using the following “keeps”:

--keep-last --keep-daily 12 --keep-weekly 6 --keep-monthly 6 --keep-yearly 1

Am I right in thinking this will keep the last snapshot, the last 12 daily snapshots (including the one kept by last), the last 6 weekly snapshosts (again keeping the two included in the daily keep), the last 6 monthly snapshots (including previous) and the last one yearly snapshot?

Is any of that completely redundant?

Also, while I’m looking at this - what does rebuild-index do and what’s the advantage of doing it?

I think that will work fine. Multiple snapshots at roughly the same time are not very costly. I run my backups daily and my script only runs forget once a week since having an extra 7 snapshots is cheaper than doing too many prunes.

That is a command for dealing with partial corruption of the index on the server. Or in some cases updating to newer versions of the software. In the backup repository, you have index files that list all the items in your data packs. This just reads all the data packs and recreates the index. So not normally needed.

Looks good. I think it’s a good idea to run restic forget with the --dry-run option, this will tell you which snapshots it would remove when you run it without --dry-run. You can find an explanation of how restic forget works here: https://restic.readthedocs.io/en/latest/060_forget.html#removing-snapshots-according-to-a-policy

@fd0 (sorry to bother) I have a question about forget and didn’t wanted to open a new topic so I searched for one similar of what I’m about to ask.

In my repository I added a new backup path just for one time. I added it this way so I can identify easily the snapshot containing this particular data. I’m not planning on backing up again this path but I do want to keep this specific snapshot forever. I used --host to rewrite the host and --time so it will be at the top of everything and with a different host name OldBackup. So, if I want to keep this specific snapshot forever, do I need to use a tag and use --keep-tag OldBackup, for example, (or maybe --keep-last 1) so it doesn’t get deleted eventually? Again, this is just one snapshot without any plan to do another snapshot for this same location ever again and my keep policies are as follows:

--keep-hourly 8
--keep-daily 7
--keep-weekly 4
--keep-monthly 12
--keep-yearly 10

As I understood, due that I used --host and it is a new path anyways, this snapshot will live for 10 years in my repository, is that correct?

It will keep 1 or 2 of the same snapshots as the daily keep, depending on the day of the week, because 12 < 14.

If you’re not specifying --group-by to restic forget then yes, because the default is to apply the policy separately to each host and path-set combination. --group-by can override what property combinations are used. (For example, we use host,tags.)

2 Likes

@cdhowie thanks a lot for your answer. I’m not using --group-by; all hosts share the same “keep” policies. So, in order to keep this snapshot forever, adding --keep-last 1 should be enough or it will be deleted anyways? Because if this “host” have only 1 snapshot, then this option is supposed to keep this sole snapshot, right? Or should I use a tag and use --keep-tag instead? Sorry for asking again, I really want to make sure this snapshot is never deleted but I want to keep the same policies for the rest of the hosts, and the keep policies have always confused me. Thanks again for your answer.

1 Like

In my understanding, you don’t need to change anything to keep this snapshot forever.

From the docs:

All the --keep-* options above only count hours/days/weeks/months/years which have a snapshot, so those without a snapshot are ignored.

Say that snapshot is timed at 2010-01-01, then as far as the host OldBackup is considered, 2010-01-01 is the only day/month/year in existence, and 2011, 2012, …, 2018 never happened. So, even --keep-daily 1 should keep this snapshot forever.

See also the example described in the docs.

Also,

For safety reasons, restic refuses to act on an “empty” policy.

Because that snapshot is the only snapshot in the host OldBackup, forgetting it would be an “empty” policy, so it cannot be forgotten unless explicitly specified by snapshot ID.

Finally, just FYI, --keep-last n is indeed applied to each group (default is host/path) separately. --keep-last 1 can indeed ensure this snapshot is kept forever, but it’s not necessary due to the multi-layer protections mentioned earlier.

2 Likes

--keep-last 1 would be sufficient, yes.

However, I tag snapshots I want to retain forever with the keep tag and then I use --keep-tag keep. It seems more explicit in my intention and harder to mess up later.

2 Likes

I test this creating a snapshot with--host and --time 2000-01-01 00:00:00 and it worked! It doesn’t get deleted. Even when I added a snapshot using the same --host and --time but using another path, it doesn’t get deleted because, as you said, they’re the only snapshots on those hosts/paths. I didn’t knew that. Thank you!!!

Indeed using a tag is more explicit and I also think it is harder to mess it up.

Thanks a lot for your answers! It really helps.

Indeed, that’s a safety feature for the case that restic backup fails, but restic forget still works: otherwise you’ll end up with no data in the repo at all. :slight_smile:

That’s a great suggestion from @cdhowie, I like it!

1 Like