I am running a synology NAS with 3 8TB drives and have an older NAS with the similar setup at my parents which I use as backup target for Hyperbackup.
As I would like to have a separate backup of my files with a second independent tool which can also be scripted (e.g. start a backup before running an upgrade of software in my homelab), I started using restic some years ago and are quite happy, thanks for your effort in maintaining the tool and ensuring its stability.
I use restic to store files to a local drive I periodically connect via USB and also to upload files to onedrive as Office365 offers 1TB per familiy member as part of the offering I already pay for.
That works fine for a while, but I start to outgrow the space (currently I store around 10TB on my NAS). For onedrive I could use multiple accounts to store more data (would not be sufficient to store everything, but at least the most important data). I also own multiple HDDs which I could connect to my NAS concurrenty or after each other.
I was wondering if someone came up with a solution to automatical partition data?
E.g. having a single backup command which creates a single index and copy the files to different targets due to certain criteria like available space.
Right now I could manually select folders and back those up in chunks to a destination until it reaches its limit and switch to the new target. But if old folders are growing, it might require a second split and that also complicates the restore as I need to find the right repository manually and it also adds the risk of forgetting to include some data. I also need to leave unused space in order to run future backups for slighly changed data. Overall the mental overhead is not great and it is also not that easy to explain to others.
Last time my single 8TB USB attached disk ran out of space, I revived my first NAS and inserted the 8TB and another old 4 TB as JBOD pool. This worked for a while but I can’t add more disks now and my storage grows constantly due to RAW photos and videos (my wife is a enthusiast hobby photographer).
Now I am thinking about buying a new disk with at least 16TB, but the problem will hit me sooner or later again.
Also, I try to reuse my old gear whenever possible, as I don’t want to waste too much money or resources for my secondary backup.
I guess the recommended approach would be to use scalable object storage, but that tends to get expensive for 10+ TB as well… not to forget potential egress costs…
Just wanted to ask if anybody has a good solution in mind, automatic splitting/resharding of repositories to multiple destinations would be quite a cool feature, but probably also hard to implement without breaking compatibility…
I am wondering if it would be possible to implement as new type of backend or maybe as part of the restic rest server itself.
I guess splitting encrypted data would be ok, if I loose one location I would expect to loose the entire repo.
Thanks for any ideas