Restic cache and rclone + google drive

i’m using restic on linux server with rclone + google drive as repo (i have shared drive with unlimited space , only 400000 files+directory limit).
so i would keep many backup, and create a script that backup then keep 365 daily backup.

works all fine, but on a server with an hdd 300gb and 200gb of mail / php files i have a cache bigger 50gb (about)

so my question is.
is better

  • edit script with --no-cache (i don’t have problem with bandwidth), but i think is a longer backup
  • mount a remote (another datacenter) “sftp/webdavs/google drive” folder and put cache in that directory?

Thank you in advance

Do not disable the cache. You don’t have a bandwidth problem but you are probably not within 1 ms of latency of the Google Drive endpoint, so this would affect performance probably in an unacceptable way, see:

I certainly don’t like the idea.

If you delete the cache and than make a single backup, is the cache already that big or does it grow to this number of time?

You could delete the cache after every backup run, this is till better than running with --no-cache as per the thread linked above.

Having a cache with low latency helps a lot! Deleting the cache after each backup may cause other problems, like always downloading non-necessary blobs but may be worth a try.

About the large cache size: Make sure you have compressed the metadata. All metadata is in json and compresses really well. Also keeping fewer snapshots might reduce the cache size - if lots of trees are changing quite often.

How do we compress cache metadata?

Just using version 2 repository and let the default --compression auto do it’s thing? Does that affect the local cache?


i wait for answer because i’m upgrading repo to version 2

but for now i run
restic -r … migrate upgrade_repo_v2
restic -r … prune
restic -r … prune --repack-uncompressed (this is running now)

but restic cache folder from 50gb now have 11gb size…
so for me is fine.

yesterday i try to use remote webdav cache , but backup keep 6 hours to finish…

better upgrade to version 2…

than you

1 Like

These two commands are enough to compress everything in the cache.

1 Like

as a bit of offtopic here, how was the migration process? I have a similar setup than you (using rclone and Google Drive) but I’m guessing the migration will make restic re-upload everything again, isn’t it? So it would be similar as creating another repo or I’m wrong in this?

i think but i didn’t check.
so i’m not sure, but i think yes…