@kapitainsky @zcalusic here is restic stats raw data…
It show 43GB, not sure why suddenly got this number
restic stats --mode raw-data
repository 010e7818 opened successfully, password is correct
scanning...
Stats in raw-data mode:
Snapshots processed: 188
Total Blob Count: 616953
Total Size: 43.000 GiB
restic version
restic 0.11.0 compiled with go1.14.5 on linux/amd64
@alexweiss @shd2h indeed, i just realize it still in 0.11!
Think i need to upgrade, but need to read first what changed to avoid data error or something for that quite upgrade.
it tells you what is your full repo size on disk/cloud
restic stats tells you what would be size of restoring latest snapshots
These numbers are never the same
LOL - yes this is ancient version.
Upgrade is very straightforward and safe. But of course good to make a backup first - especially that your repo is not big.
Hmm okay… that’s interesting, never know about it before.
So do you think if i upgrade Restic it will help to make cache smaller?
Or it will help prune faster so i can run it more often like what @zcalusic said?
Definitely upgrade to the latest restic, it contains a lot of improvements that are relevant to you.
Yes because as a minimum all metadata will will compressed. How much smaller depends on your data. And given how old version of restic you are using huge amount of fixes and improvements has been implemented in the meantime.
Re prune - first it is much faster in newer restic as @alexweiss mentioned. Second more often you run it less job it has to do. The worst thing to do is to run it very rarely - then it is slow.
Okay, thank you everyone, i will try to simulate and implement new Restic in our system.
I remember last time i did, i can’t restore backup because of different version of Restic. Basically like Restic can’t restore because snapshot created by different version with current active Restic. but let me check again soon to make it sure.
It can only happen when you migrate to V2 repo and then try to use old restic to restore. It is one way road:) You move to the new one and should forget about using old one.
Okay, so i updated Restic into last version,
restic version
restic 0.15.2 compiled with go1.20.3 on linux/amd64
after 7 days, i checked again.
restic stats --mode raw-data
repository 010e7818 opened (version 1)
scanning...
Stats in raw-data mode:
Snapshots processed: 192
Total Blob Count: 626434
Total Size: 43.852 GiB
restic cache
Repo ID Last Used Old Size
----------------------------------------
010e78184b 0 days ago 20.040 GiB
----------------------------------------
1 cache dirs in /root/.cache/restic
so it still 50% of backup size, i tried to prune.
restic prune
repository 010e7818 opened (version 1)
loading indexes...
loading all snapshots...
finding data that is still in use for 192 snapshots
[19:05] 100.00% 192 / 192 snapshots
searching used packs...
collecting packs for deletion and repacking
[0:02] 100.00% 6892 / 6892 packs processed
to repack: 16414 blobs / 1.938 GiB
this removes: 198 blobs / 1.624 GiB
to delete: 335 blobs / 1.643 GiB
total prune: 533 blobs / 3.267 GiB
remaining: 626981 blobs / 43.936 GiB
unused size after prune: 86.202 MiB (0.19% of remaining size)
repacking packs
[0:22] 100.00% 72 / 72 packs repacked
rebuilding index
[0:02] 100.00% 6800 / 6800 packs processed
deleting obsolete index files
[0:00] 100.00% 19 / 19 files deleted
removing 96 old packs
[0:03] 100.00% 96 / 96 files deleted
done
and restic cache down to 17GB.
restic cache
Repo ID Last Used Old Size
----------------------------------------
010e78184b 0 days ago 16.844 GiB
----------------------------------------
1 cache dirs in /root/.cache/restic
du -sch /root/.cache/restic
17G /root/.cache/restic
17G total
I still considering this as too big, but really not sure what should i do next.
For now my plan just to reduce backup frequency.
your cache is almost 40% size of the repo size which indeed is real big. To double check that it is correct I would just delete all cache (it is 100% safe) and see if during subsequent backups/prune it returns to this value.
There are many people requesting cache limit option to be added:
But it is only discussion at the moment.
Yeah, that what i do, deleting cache folder regularly, because in 2-3 days it will be back at 16GB and start growing few mb each day (i did create backup 2 snapshots daily, and delete all snapshot older than 30 days).
I even thinking to put in cron to delete cache regularly 
btw, if I put --no-cache in all my Restic command, will it mean cache directory (/root/.cache/restic) won’t created at all? i know it will has drawback, but in my case, it will save some storage for it.
yeah it will run without any cache - it will be much slower - but if it is acceptable it might be an option. You have to test.
BTW - data you backup - is it changing a lot? I am trying to understand why such large cache.
so how come your repo has 192 snapshots? Not that it is an issue. People have 100s of snapshots and all works fine.
so how come your repo has 192 snapshots?
Yea, I just realized it too now, but it’s actually backup of website file, database and webserver configuration. So when doing backup, it will create 3 snapshot at once.
BTW - data you backup - is it changing a lot? I am trying to understand why such large cache.
website file might not really changing a lot, but database that might changing frequently.
What is the output of find path/to/repository -size +30M? Is there maybe a folder in the backup which contains tens of thousands of files (directly inside the same folder)?
You could upgrade the repository to format version 2, which supports compression. That typically reduces the metadata size be a factor 3 or more. (I know that this is not a real solution.)