First of all congratulations for this magnificent piece of software! It’s a great discover!
I’m checking the shiny new feature “check” of the 0.9.0 release. Like the developer says, it creates a dir in a temp directory in order to not stress remote backend with petitions. My problem is that “/tmp/” partition only has 5GB size, and my remote repo ~800GB. What happens? restic fills up “/tmp” partition because it creates a temporary directory > 5G:
# ./restic_0.9.0_linux_amd64 check
using temporary cache in /tmp/restic-check-cache-260276770
repository 36ab3d95 opened successfully, password is correct
create exclusive lock for repository
load indexes
check all packs
check snapshots, trees and blobs
[...]
And my question is: would be possible to specify this temporary directory in another location?
Thanks for trying restic! You can set the path for the local cache used during check with --cache-dir ~/restic-cache (EDIT: and you also need to pass --with-cache). For normal commands like snapshots and backup, restic uses a cache in the default location (e.g. ~/.cache/restic). But for the check command this cache isn’t used so that it can detect when files are missing or corrupted in the repo. But without a cache, restic check is really slow, so we decided to use a temporary location for it and load all data fresh from the repo, but cache it locally in case restic needs it again (which is often the case).
There’s no built-in way yet, sorry. You can remove the cache and restic will rebuild it on the fly, storing only data that is strictly needed, so it’ll be smaller than before. Over time, it’ll fill up again though, and only shrink when you remove data from the repo e.g. via forget/prune.
I ran into the same issue today. /tmp is on tmpfs here, so it’s only 2GB and it quickly filled up when running restic check. Unfortunately --cache-dir didn’t seem to work here, restic still used /tmp
Ah, sorry, I had a look at the code again and the twist is that the cache specified with --cache-dir is only used for check when --with-cache is specified:
Correct. The current implementation works as follows: restic check by default does not use the regular cache in ~/.cache/restic, so that it detects corrupt files stored on the server. You can change this by passing in --with-cache, which will use the regular cache. If the regular cache is set to a different directory via --cache-dir, restic will only use that directory when --with-cache is specified. Thinking about it, I came to the conclusion that this is necessary: if you have your regular cache at /srv/backup/restic-cache, you wouldn’t want restic to use it during check unless --with-cache is specified.
Instead, restic builds a new cache in a temporary directory. You can use TMPDIR to globally set the temp directory for restic (the environment variable is evaluated by the Go stdlib, not a restic thing), so restic will create the temporary directory for the cache there. Or you can manually set the directory with --cache-dir, but then you need to pass --with-cache to enable using the directory.
I don’t like that it’s so complicated to do right now, so if anyone has an idea on how to simplify this, please let me know
Hi, so has anything changed regarding this in 0.9.2?
I think if I run restic check/snapshots/stats by just passing --with-cache (e.g. restic stats --with-cache > $LOG_FILE) it won’t use the local cache because I didn’t specify a --cache-dir, right?
Or, will it use the default local cache, that I see at ~/Library/Caches/restic/ (~8MB data for ~5GB data set to backup and stored on B2) on Mac?
restic uses a cache in the default location (e.g. ~/.cache/restic )
There’s nothing related to restic in ~/.cache/ btw. Is it expected on Mac?
Also, I see only --no-cache and --cache-dir options for backup so I guess there’s no way I can do backups based on cache and avoid some B2 API calls during the backup itself. Or, is --cache-dir implicitly means --with-cache? What if I don’t pass --cache-dir nor --no-cache – will it use the default cache location for Mac ~/Library/Caches/restic/?
My purpose:
use local cache (if available) for anything during four times daily backup runs (avoiding as much B2 API calls as it can other than actually uploading the changed files) – I do backup, then check, then stats.
Do restic stats, snapshots, and “forget with --prune” without local cache during once a week maintenance runs.
Uhm, the stats command does not have --with-cache, which is an option to the check command.
in general, when you set the cache dir manually with --cache-dir (which is a global option all commands have) then restic will use that directory.
Yep, the directory on macOS is ~/Library/Caches/restic.
I don’t understand that sentence. restic uses a cache by default unless you pass --no-cache. You can configure the location of the cache with --cache-dir. There’s no explicit way to enable the cache, it is always on unless disabled.
So, if you always want to use the cache, you don’t need to do anything besides restic check, for which you need to pass --with-cache.
