Estimation of number of read operations required by check operation


#1

I thought I had read somewhere that the check operation requires about 4 read operations per file but it seams I can not find that information again. I might have just dreamed it.

Anyway, I have just run a check operation on my b2 backup with ~66K files and it required ~110K class B b2 operations. This operation failed yesterday because my daily cap was set to low.

In order to set a sensible limit value, Is there a way to get an estimation about how many of these operations the check procedure will require ? Does it mainly depend on the number of files or does the number of snapshots also have an important impact ?


#2

Oh good! I am about to do tests like this on a 100k file repo on B2. I’ll try to keep some stats and share them here.

A $2 daily cap is 5 million Class B operations, so you can probably afford to safely overshoot. For your example ‘check’ you’d need a daily cap of $0.05 :slight_smile:


#3

I did another test with a another bucket with slightly more snapshots, but at least an order of magnitude less files (only ~100MB backup) and the check operation required about the same number of class B operations as on my 38GB bucket.

I have of course raised the cap to be on the safe side, but I am worried about not being able to estimate how much the requirement of this class of operations is going to raise as I add snapshots. If you reach the limit you loose the ability to work with any of your buckets as it is set account wide. What will happen when I run a check operation next year ?


#4

It is a worry, but if you enable alerts in B2, you’ll get a email/page at 75% of quota and can adjust. If we could plug the B2 API in as a source for Nagios/Prometheus etc. then we could alert when the trend predicts exhaustion. Though I haven’t spotted an existing Nagios/OMD plug-in for this.

One option with B2 is that you can create multiple accounts and use the Business Groups feature to invite and join them into the same billing group. Each user has their own buckets and daily caps for those buckets; which addresses your concern. With the Business Group you can set the caps, view buckets, and pay fees from the master account. The only hassle is manage multiple logins for the users, though you never really need to log into those accounts.