Which is the cheapest storage to use restic backup? Using the prune operation daily.
Regional or Nearline or Coldline (google storage)
The answer probably depends on your exact needs, and I suspect we won’t be able to help you without any information at all 
I don’t have any experience with GCS, but I’ve heard that B2 is pretty cheap if you don’t run prune so often. Maybe it’s cheaper to not run prune all that often and keep the data at B2?
1 Like
the improvement of the prune operation will provide lower cost?
Wow, that’s very frequent. Do you have a lot of data churn? Why do you need to run prune every day?
because is a new database dump every day… about 10gb each dump
Keep in mind that nearline has a 30 day minimum storage duration, and coldline has a 90 day minimum. If you transition an object to nearline, this means you commit to pay for 30 days of storage, even if you delete the object before then.
A minimum storage duration applies to data stored as Nearline Storage or Coldline Storage. You can delete the file before it has been stored for this duration, but at the time of deletion you are charged as if the file was stored for the minimum duration.
If you are running prune daily, this means that indexes will be rebuilt daily. Therefore, you should keep indexes at regional.
Pruning may also rewrite packs, which uploads a new pack and deletes the old pack. If the old pack was not stored for the minimum duration, you will still be charged as though it were.
Therefore, I would strongly advise against using nearline or coldline storage for restic. Whatever you save in storage costs may easily be eaten up by early deletions, and it could cost you much more in the long run, unless you prune every 2-3 months.
(Note that the situation is the same for the S3 infrequent-access (IA) storage tier; there is a 30-day commitment for objects transitioned to IA.)
2 Likes
I went through this decision process recently for a different use case, read this thread, and decided to use Nearline. I back up ~100 GiB daily but prune only a couple times a year. After months of use 99.9% of the costs for me are byte storage and multi-region replication, meaning restic didn’t add any significant overhead with Class A and B operations. If any others want to repeat how I double-checked costs, it was straightforward to export billing data to BigQuery, wait a day for it to backfill, then run a BigQuery query like:
SELECT
project.id project,
sku.description sku,
usage_start_time,
SUM(cost) cost,
STRING_AGG(DISTINCT currency) currency,
SUM(usage.amount) usage,
STRING_AGG(DISTINCT usage.unit) usage_unit,
FROM `<table_name_created_by_billing_export>`
GROUP BY 1, 2, 3
ORDER BY 1, 2, 3
@m-d-brown Thanks for your remark! Note that this thread is 5 years old and the prune command has been completely reworked since and is much less problematic with respect to data access.