Brand new Restic user here. I’ve just set up a B2 repository and backed up about 2GB of data. I then ran a few add’l backups of the same data with very little changes as a test. The transactions after this in backblaze are as follows: Download: 1G, Class B: 5795, Class C: 137. The download and the class B transaction are higher than I was expecting. Is there something I can do to limit this?
2 Likes
Welcome to the forum, and thanks for giving restic a try!
I guess that you’ve used the latest released version 0.7.3, is that correct? With that version you can use restic backup --force to create a new backup, which instructs restic to not load any metadata for a previous backup from the repo. That limits the download transactions. The downside is that it will re-read all files locally, because it knows which data is in the repo, it just doesn’t know which data the files contain without old metadata.
The other things you can try is running the code from the master branch. Either build it yourself (install Go, clone the repo, run go run build.go) or download the top-level release from here: https://beta.restic.net/?sort=time&order=desc
Since a few weeks the master branch contains code which adds a local cache for all metadata (and is not contained in any released version of restic yet) so that once the cache is primed, the B2 backend isn’t used for loading metadata. For existing repos you may need to run restic prune once to use the cache to its full potential (or start over with a new repo).
1 Like
I have a related question, so I’ll just tag along (and I am a new restic user as well):
I have backuped about one GB of data to B2, around ~190 files. After a first backup, I ran restic check, which stopped because it ran out of allowed b2_download_file_by_name calls and I am at 7206 calls afterwards.
I am running restic 0.7.3 (v0.7.3-195-g8d37b723), compiled with go1.9.1 on freebsd/amd64.
Is this normal behavior (using many more downloads than there are files for a check)?
Then also for you: Welcome to the forum and the restic community!
As a backup program restic is optimized for safety (and security), not so much for minimizing backend requests. So we’ve made the decision to not use the cache for the check command by default, so that you really check what’s in the backend (and not only the local cache, which may not help if the hard disc dies and you want to restore).
You can use the cache for check with the --with-cache flag. Then it’ll pull down the files containing the metadata and run the checks on them, which limits the number of queries to the B2 API to the number of files in the repo. You could also use a different cache directory with --cache-dir /tmp/restic-check-20171024 which forces restic to really download the files.
What happens here is that restic checks that all data structures referenced in all the snapshots are actually there, and without the cache it’ll load a small part of a file for each directory in each snapshot, that will be much more than the number of files.
The top level beta release worked on my windows machine, however on my mac I’m not sure how to execute it. I downloaded restic_v0.7.3-237-g06bd606d_openbsd_386, I am assuming that’s executable? I received a permissions error so did a chmod, but now it says “cannot execute binary”. Is the file an archive of some type? I do not see an extension.
That’s the wrong binary, for OS X the architecture is named darwin, so you need the _darwin_amd64 binary.
So, if I were to start off a fresh B2 bucket, to limit my transactions, I should start off with the latest beta containing the local cache functionality? Any specific flags to enable this feature or it’s done by default?
It’ll work out of the box, no need to configure anything. Let us know how it goes!
Initial backup final completed:
scanned 3583 directories, 67314 files in 0:05
[40:42:48] 100.00% 2.479 MiB/s 354.871 GiB / 354.845 GiB 70898 / 70897 items 0 errors ETA 0:00
duration: 40:42:48, 2.48MiB/s
I’ll be testing a scheduled task in the next while.
restic 0.7.3 (v0.7.3-237-g06bd606d)
compiled with go1.8.1 on windows/amd64