I found two refrences in forum →
Are there any problem associated with live system backup? I read the above sources but would still like to confirm and also request to please add the related materials in documentation.
Can you elaborate what you mean by “live system backup”? Generally there is no technical problem backing up a system being used from restic’s side. The trouble usually starts when your system runs transactions touching multiple files and/or a database.
In that case you need to make sure that restic finds a state to backup that will work again when you restore it. In reality that usually means briefly stopping the system in question, dumping a database, doing the backup and then starting the system again.
Alternatively, instead of keeping the system stopped during backup, you might consider taking a file system snapshot and backing that up instead.
1 Like
and to add to the above:
The potential “problem associated with live system backup” holds for any backup solution and is not specific to restic in particular.
1 Like
Of course, good point! Thank you.
As already mentioned worse case it can lead to wasting time and money as your backup can be potentially useless.
More “live” your data is more likely you will end up with something you do not need:) You have to asses yourself what exactly you are backing up and what are chances that data can change during restic operations. And what it means in terms of restoring such patchworked backup.
To give you some feeling what I am talking about let’s imagine two drastically different scenarios:
- Directory containing your holidays pictures. Most likely nothing will change when restic is ploughing through it. And even if yes then worst case some pictures might be missing. Backing such “live“ data is pretty much safe. But not 100%.
- Directory containing very busy database - constant transactions keep changing files all the time. Backup up such data from “live” system most likely will lead to backup which won’t work when you restore it. Backing such “live“ data is pretty much impossible without some extra steps.
So think what your data is and choose your strategy accordingly.
Here some good read about this subject. It sheds more light on why and how.
Well designed backup solution (which restic is only part of) should always use stable source data. There is no if or but.
Myself I always use filesystem snapshots as it is cheap (does not use any space) and fast. But if you are working with static data (like Documents/Pictures/Films folders) you can safely live without it. Unless your data is mission critical for whatever reason:)
Can you elaborate what you mean by “live system backup”?
For now, i am just using laptop runnign ArchLinux as daily driver. Have bunch of scripts which time to time read from sqlite database and write changes to it. By live-system i meant a system which can be used during the backup operation.
More “live” your data is more likely you will end up with something you do not need:)
Directory containing very busy database - constant transactions keep changing files all the time. Backup up such data from “live” system most likely will lead to backup which won’t work when you restore it. Backing such “live“ data is pretty much impossible without some extra steps.
I am a little bit confure here, i don’t care if data is old, i.e the backup started on a file but just a few ms later the file got update with new content. What matters for me is that the file that got backed up is not in corrupt state, i.e messed up. Does modern fileSystem provide no such feature ( i am on btrfs ), where a program once accesses a file, even if file content changes, it can still access the old content which it was initially accessing.
On another note, if the above is true where backed up file gets corrupted becaues it got updated during backup, is there way to figure out those files in the restic repository, so that i can think of another way to backup them in future.
Myself I always use filesystem snapshots as it is cheap (does not use any space) and fast. But
I was concerned with BTRFS snapshot in the scenario where the data block that snapshot is referencing gets updated in that case, the old will contribute to disk space and this is problematic for cache files and there are many in $HOME. But i think i can just take snapshot and backup then delete the snapshot.
BTRFS provides means in form of filesystem snapshots to achieve it. But it is something you have to take care of. restic does not. You have example in the link I provided. It is for ZFS but for BTRFS it will be pretty much the same in principle.
Deleting snapshot immediately after backup is good option if you are limited by space or your data changes a lot. Other option would be to relocate caches to dedicated BTRFS subvolume where even you can turn all COW magic off - similarly like for swap.
Thank you for that article, it answered all my doubts.
In the restic docs there’s no mention of taking backup of file system snapshot, is the live system backup not a problem for majority of people?
Majority of people probably never attempted to restore all data into a new system. And most likely only care about holiday pictures:)
But seriously hard to tell what majority of restic user sdo. And whatever they do it has nothing to do with what is the right way:)
1 Like