S3 bucket storage size bigger than what restic stats reports

Getting Help
1

We have our own S3 server (NetApp StorageTec) which I use with restic.

One of my Linux servers uses more S3 storage than what restic stats reports (21 vs 9 GB):

root@sp-i:~# restic stats --mode raw-data
SG_TENANT=tik-sp
RESTIC_REPOSITORY=s3:s3.tik.uni-stuttgart.de/tik.sp-i.restic
repository afa70d4d opened (version 2, compression level auto)
[0:00] 100.00%  6 / 6 index files loaded
scanning...
[0:01] 100.00%  12 / 12 snapshots, 235713 blobs, 9.188 GiB
Stats in raw-data mode:
     Snapshots processed:  12
        Total Blob Count:  235713
 Total Uncompressed Size:  25.976 GiB
              Total Size:  9.188 GiB
    Compression Progress:  100.00%
       Compression Ratio:  2.83x
Compression Space Saving:  64.63%
root@sp-i:~# 
root@sp-i:~# s3cmd du s3://tik.sp-i.restic/data
 21319520953    1296 objects s3://tik.sp-i.restic/data
2

Do you have some sort of retention policy that’s keeping deleted files?

Also, if you’ve run “forget” but haven’t run “prune” then I’m pretty sure (please correct me if I’m wrong here) that --mode raw-data won’t show anything forgotten.

3

For S3 policy I have to ask my S3 admin.

I always run prune after forget.

root@sp-i:~# s3cmd ls -r s3://tik.sp-i.restic/data/ | wc -l
1296

Is there a restic command that shows the number of S3 objects?

restic stats only reports files, not S3 objects.

4

this was THE hint!

Indeed I have run “restic forget --prune” and not “restic prune” which is a different command.

Now:

root@sp-i:~# restic prune
SG_TENANT=tik-sp
RESTIC_REPOSITORY=s3:s3.tik.uni-stuttgart.de/tik.sp-i.restic
repository afa70d4d opened (version 2, compression level auto)
[0:00] 100.00%  6 / 6 index files loaded
loading all snapshots...
finding data that is still in use for 12 snapshots
[0:01] 100.00%  12 / 12 snapshots
searching used packs...
collecting packs for deletion and repacking
[0:00] 100.00%  1296 / 1296 packs processed

to repack:         32387 blobs / 372.741 MiB
this removes:      11493 blobs / 330.714 MiB
to delete:          6728 blobs / 9.852 GiB
total prune:       18221 blobs / 10.175 GiB
remaining:        241015 blobs / 9.671 GiB
unused size after prune: 494.619 MiB (4.99% of remaining size)

repacking packs
[0:12] 100.00%  98 / 98 packs repacked
rebuilding index
[0:00] 100.00%  7 / 7 indexes processed
[0:00] 100.00%  7 / 7 old indexes deleted
removing 690 old packs
[0:24] 100.00%  690 / 690 files deleted
done

root@sp-i:~# s3cmd du s3://tik.sp-i.restic/ | d3
10,405,136,244 628 objects s3://tik.sp-i.restic/

This was my fault!

Thanks!

5

You got me confused now. Running restic forget --prune should be the same as running restic forget and then restic prune. At least that’s what the manual says and it’s also what I’m usually doing.

6

There is a subtle difference!

--prune                             automatically run the 'prune' command if snapshots have been removed

So if you run the same forget command and it doesn’t remove anything, then prune won’t do anything.

But yes whenever I use the forgot command I always give it --prune as well so I don’t end up in the above situation.

7

Well yes but if you don’t forget anything, there should be nothing to prune. Ah anyway. Good thing you got it sorted out :smiling_face_with_sunglasses:

8

Yea so consider this:

restic -r repo forget --keep-weekly 4 --keep-monthly 12 --keep-yearly 10

And let’s say it removes 10 snapshots.

If you then do

restic -r repo forget --keep-weekly 4 --keep-monthly 12 --keep-yearly 10 --prune

Nothing will get pruned because you didn’t forget anything new.

restic prune forces a prune operation. restic forget --prunewill only run a prune if something was forgotten.

Personally I wish the logic was changed so that --prune was the default and you have to give the flag --no-prune for prune not to be run after every forget. But changing it now would break many workflows I’m sure :slight_smile:

9

Interesting. I wasn’t aware of that difference. Thanks for writing this! In practise it probably makes little difference for me but it’s good to know!

10

Really? That’s a surprise to me. But yes, the documentation says so.