Prune job won't finish, eat up memory

Getting Help
1

hi and hopefully someone can help me.

our customer makes daily and hourly backups of multiple hosts (a lot oh backups).
since 1 month, the prune job won’t finish. we have 2 restic server as long and short time archive.
both systems have the same problem. sometimes die with oom.
actually, we have 3600 snapshots.

i tried the index rebuild, after seeing some errors in the log.
the strange thing is, that there is nothing to delete.
forgot has no jobs marked for delete.

we are on a very old version and i try to upgrade today.

but, do you have any idea, where to start to debug the problem?

many thanks for helping …
ronny

2

Without some basic logs, this is no more than a sad story I’m sorry :slight_smile:

There’s a reason when this pops up when you click on the new post button, and it’s not to give you something to select all and delete!!

<prefilled text>

If you need help with a problem, please always include (in your post below this comment section):

• The output of restic version.
• The complete commands that you ran (leading up to the problem or to reproduce the problem).
• Any environment variables relevant to those commands (including their values, of course).
• The complete output of those commands (except any repeated output when obvious it’s not needed for debugging).

3

hi and sorry for that, here the data
version:
restic 0.9.4 compiled with go1.11.6 on linux/amd64
now upgraded to
restic 0.14.0 compiled with go1.19.8 on linux/amd64

cmd:
restic -r /srv/backup/restic -p /root/.restic --cache-dir /srv/restic/cache/ forget --keep-hourly 21 --keep-daily 730 --keep-weekly 0 --keep-monthly 0 --keep-yearly 0 --prune

output:
killed oom, with 40gb ram and 70gb swap

sorry again,
ronny

4

A couple of things to try that I believe will help

set the GOGC environment variable to 5.
This means the Go Garbage collector runs much more often and can help with memory use.

Also, upgrade to a much later version of restic, 0.17.1 is out and there have been more improvements in memory usage for prune operations.

5

thanks for answer
i upgraded the first system and now the prune job give us:
ERROR: decrypting blob … ciphertext verification failed

i will test and look forward,
ronny

6

As it was already mentioned: