Simple question, does restic keep an SQLite file somewhere containing snapshots data containing, ID, host, tags, path, timestamp ?
If this exists where to find it ? And finally wouldn’t it be nice to have it knowing the list of files issued by
$ restic diff ParentSnapID CurrentSnapID ?
Nope, it does not. All of this information is stored in the repository, see restic Design Document - Documentation for restic for more details.
What do you mean by this, considering that restic’s diff command outputs a list of files - seems exactly like what you are asking?
1 Like
Yes restic diff does what I’m looking for but having a local SQLite remembering what happend on each snapshot is faster than asking the repo.
That’s because I’m planning to do a bash watchdog script that triggers alerts (mails) according to conditions, for example, if a hourly backup has not been done for two hours then alert. Why do that ? Cause my restic is running on laptops and laptops are often disconnected.
I’ve used Duplicati for a short period of time which AFAIK uses SQLite to save a local copy of file information. With a certain amount of snapshots and files in the repository it literally took minutes to display and browse the contents of a directory.
Using a database isn’t necessarily faster.
1 Like
Actually I’was not thinking to keep the result of
$ restic ls -l latest
for each snapshot but rather the result of
$ restic diff previousSnap latestSnap
which (in my case) makes only a few files for each snapshot.
restic does not do that by default since it does not need this data. You can easily implement this yourself though in a shell script.
What’s your use case? What are you trying to do?
I’m trying to implement a watchdog bash script for my backups, I have four Macs, on each Mac runs an hourly backup for user directory (if some files are changed), it also runs a complete host backup daily during the night.
As my uplink is really slow, When one backup is running, the other waits the first one to finish.
What I would like is some watchdog doing some stats on my backups and alert me if, for example, the hourly backup had not been done for…,say 5 hours. Same for the daily one, if it’s late of say 6 hours then email.
I was planning to parse the result of $ restic backup in an SQLite database cause I feel it would be easier to check on it than keep the data in a flat file. It will also offer to get some statistics (time between backups, mean time of backup,…).
Going further (but I don’t know if this make sens), I was planning to keep the result of
$ restic diff previousSnap currentSnap in another SQLite database which, if needed could be linked to the first one to know what files were backed up during a backup, or even find the history of a file.
I’m still thinking about the best way to do that.
If someone has better ideas, I’m listening.
Just giving my thoughts and I precise that I’m far from succeeding…
I’m planning to have two SQL databases, one contains the result of
$ restic snapshots
say ResticSnaps.db
ResticSnaps.db has one table with columns
ID
Date
Host
Tags
Directory
but is also amended with data parsed from
$ restic backup
output each time a bakcup is done
i.e. columns
DateBegin (my script begin time and date)
DateEnd
Newfiles
ModFiles
UnchagedFiles
NewDirs
ModDirs
UnchangedDirs
QuantAdded
ProcessedFiles
ProcessedQuant
ResticDuration
the other database contains the result of the
$ restic diff PreviousSnap CurrentSnap (of same host and dir)
command.
Say ResticBacks.db with a table and columns
ID (snapshot ID)
FileDir
FileName
Status (M, + or -)
ResticSnaps would be backpup up on repo while ResticBacks.db stays on local machine (can be also backup but only in a security perspective).
Each time needed, ResticSnaps.db would be restored from repo and amended with last $restic backup data.
Then I could do some stats on the database.
This way each machine could not only monitor its own backups but be a watchdog for others.
In case one host is down, others could do an alert (send a mail for example).
If ever needed, linking ResticSnaps with ResticBacks on snapshot ID as foreign key could give us what happend during a snapshot (faster than diff) and the other way around, looking from the file perspective, it could give us a file history.
Feel free to comment…
For my project, it would be better if restic waits until another backup is finished.
Is there any way to check if another machine is cuurently backing up (I’m using wasabi S3) ?
Question 2 -> How do restic choose the parent snapshot ? Can I consider parent as the previous backup for same host and path ? Can I rely on parent to check time between backups ?