How can I create a new snapshot only if the destination folder changes

Getting Help
1

A nice project.
I wrote a script to run restic every hour. I just want to create a new snapshot when the destination folder is different from the last snapshot. But even if the destination folder has not changed, a new snapshot is created each time.
What flags can do this?Create a new snapshot only if the destination folder changes.

2

What do you mean by the destination folder? There is the repository, which would be the closest thing to a destination folder, but presumably you aren’t interested in looking at when that changes because that wouldn’t make sense. Then there’s the source folder and/or files, which is the ones you ask restic to back up. Please explain what you mean.

Yes, restic creates a new snapshot every time you run a backup, but note that it will not back any data up that has already been backed up. So if you run the backup again without any files having changed, no new data for those files will be saved in the repository (this is called de-duplication). Restic will just spend a little time scanning the files it needs to, but that’s about it. So there’s no harm in this.

Restic does not have any flags or options to run a backup only when certain files changed - that is outside of its scope and something you have to configure outside of restic. There are filesystem watchers that might help you, so you only run the backup job when certain files or folders change.

Finally, what problem are you trying to solve by wanting to do what you describe here?

1 Like
3

Your answer is very clear. Thank you.
“Destination folder” is the wrong expression, I mean source folder and/or files, which is the ones I ask restic to back up.
I simply run restic every 1 hour to back up my files. Most of the time I don’t make changes to the file, so there will be a lot of new snapshots that are the same as the previous one. If restic has its own “flag” and can find that it is the same as the previous snapshot, it will not create a new snapshot, which is too concise.

5

In case of Linux I would really give inotify a try for that.

6

Yep, you are entirely right :slight_smile: But, how is this a practical problem?

When it works like it currently does, you can just forget about it (besides checking your backups now and then). The “unused” snapshots don’t occupy any relevant space in the repository, and unless you have a big problem with how long the scan takes during those backup runs, I really don’t see the problem.

If you want to restore a file, you can just search for that file using restic find if you don’t know a point in time when you last had the file in a known good condition.

Also, it’s good practice to regularly apply restic forget policies, which will regularly clean up the list of snapshots.

It seems to me like you’re trying to solve something that doesn’t need solving :slight_smile:

1 Like
7

The idea I had to enable this was to:

  1. make a snapshot
  2. query restic for the last two snapshots
  3. run restic diff on these two snapshots
  4. if $(( $added + $removed)) == 0, then run restic forget

However, I am not a skilled user of restic. I just wanted to use it to define a unique cryptographic hash for specifying one unique configuration of my data so that me and my colleagues can reanalyze old data reproducibly using code from the same git commit and data from the same restic hash as was used to develop the original analysis. When there are multiple hashes referring to identical data sets, its not so easy to work with these hashes with others.

However, the inotify suggestion sounds like an even better solution where it is available. One advantage of the inotify approach is that it doesn’t have to fight with restic unlock. So, you can keep using restic for mounting backups while still snapshotting it.

8

Thank you for all your patient answers and inspirations.

I am not a skilled user of restic either. According to Kyle’s idea, I wrote a windows .bat file. It’s ridiculous, but it works.

It can solve the problem of adding too many same snapshots. Although the same snapshot does not take up any space, for me, I can see the changes intuitively, and it is convenient to restore to the snapshot before the change.

It seems hard to find a filesystem watcher in windows system.

**Thanks for rawtaz betatester77 Kyle Let me like resitc more.

I have written RESTIC_PASSWORD in the environment variable before

@echo off
setlocal enabledelayedexpansion
set resitcpath="G:\autobackup\kee"
rem your repo path
set /a allrow=0
set /a row=1
set /a diffrow=0

for /f "delims=" %%i in ('restic -r %resitcpath% snapshots') do (
set /a allrow+=1
)
rem sum how many rows output

set /a lastrow=%allrow%-2
set /a prerow=%allrow%-3
for /f "delims=" %%i in ('restic -r %resitcpath% snapshots') do (
if !row!==%prerow% set prerowid=%%i
if !row!==%lastrow% set lastrowid=%%i
set /a row+=1
)
set idpre=%prerowid:~0,8%
set idlast=%lastrowid:~0,8%
rem find lastest two snapshots and get IDs

for /f "delims=" %%j in ('restic -r %resitcpath% diff %idpre% %idlast%') do (
set /a diffrow+=1
echo %%j
)
rem sum how many rows restic diff output

echo %diffrow%
if %diffrow%==8 (
restic -r %resitcpath% forget %idlast%
)
rem if restic diff output rows is 8 than shows nothing changed.Use resitc forget to delete the newest snapshot.
9

Please be aware that when there’s no diff in the snapshots, then they take almost no space at all in the repo. So you don’t gain that much by removing them :slight_smile:

10

Glad to meet u.
Sometimes, I want to restore from a latest different previous snapshot. I don’t know what files are changed.I use script to create snapshot per hour. How can I fast locate the snapshot?

11

You can try using restic find <filename>, it’ll show you which snapshots have the files you need.

2 Likes