Incosistent behavior with restic when current directory changes for the command

I see some inconsistent behavior, at least in my opinion when restic repository are moved around, and we use relative paths to restore the backups.

Here is a test.

I  ~/test_restic ❱ mkdir repo src restore
I  ~/test_restic ❱ echo test > pass
I  ~/test_restic ❱ restic -r ./repo init -p ./pass
created restic repository 78040fd969 at ./repo

Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is
irrecoverably lost.
I  ~/test_restic 2.8s ❱ restic -r ./repo -p ./pass backup ./src
repository 78040fd9 opened successfully, password is correct
created new cache in /home/babu/.cache/restic
no parent snapshot found, will read all files

Files:           0 new,     0 changed,     0 unmodified
Dirs:            1 new,     0 changed,     0 unmodified
Added to the repo: 400 B

processed 0 files, 0 B in 0:00
snapshot e99833a4 saved
I  ~/test_restic ❱ restic -r ./repo -p ./pass restore latest --path ./src -t restore/
repository 78040fd9 opened successfully, password is correct
restoring <Snapshot e99833a4 of [/home/babu/test_restic/src] at 2022-01-14 15:07:51.626207528 -0800 PST by babu@babu> to restore/

So far so good. However, let’s see what happens when we move the repository to a different location and try to restore.

I  ~/test_restic ❱ mkdir subdir
I  ~/test_restic ❱ mv repo subdir
I  ~/test_restic ❱ cd subdir
I  ~/t/subdir ❱ mkdir restore
I  ~/t/subdir ❱ restic -r ./repo -p ../pass restore latest --path ./src -t restore/
repository 78040fd9 opened successfully, password is correct
latest snapshot for criteria not found: no snapshot found Paths:[./src] Hosts:[]

Upon further investigation, if the current directory from which the restore command is executed changes (different from where the backup command was executed), restore does not work anymore with relative paths.

This behavior seems inconsistent to me, at least when we compare to tools like tar. Is this the expected behavior? Thanks!

Hi, welcome to the forums.

Which paths does restic say if you’d like to list snapshots in the repo?

You’re backing up with ./src, which your shell will expand, and as a result restic will backup /home/babu/test_restic/src (as it already wrote in the output).

But while restoring, if you’re on different folder, you shouldn’t give ./src because it’ll expand to “current folder + /src” which is not a path within the backup.

I didn’t try it with tar, but this doesn’t look inconsistent, imho.

// edit: Now played around with it, and looks like it also works as you wanted. I am not sure if I am doing something different or not. Which version of restic you’re using? :eyes:

Hi,

Thanks for the response. I am using restic version 0.12.1. Thanks!

As far as I know shell does not expand paths unless it has a ‘*’. Just to make sure, I quoted the path. Hence instead of ./src, I used "src" in the command line. I still see the same behavior.

I also just wanted to update with the behavior of tar here.

I ~/test_tar> mkdir src 
I ~/test_tar> echo test > src/test.txt
I ~/test_tar> tar cvf src.tar src
src/
src/test.txt
I ~/test_tar> mkdir extract
I ~/test_tar> tar xvf src.tar src -C extract/
src/
src/test.txt

As we can see in the above case, we can specify a relative path and make tar extract the files from it.

I ~/test_tar> mkdir subdir
I ~/test_tar> mv src.tar subdir/
I ~/test_tar> cd subdir/
I ~/t/subdir> mkdir extract
I ~/t/subdir> tar xvf src.tar src -C extract/
src/
src/test.txt

Also, even if the file location changes and if we try to execute from a different directory we still get the same behavior.

Thanks!

Ah you also move the repository, I’ve missed that sorry. Also tried to test it but I think I am still doing something different:

~> echo $RESTIC_REPOSITORY
/home/gurkan/Desktop/zz
~> restic backup ./Zomboid # backing up with relative path
repository 9953871b opened successfully, password is correct
no parent snapshot found, will read all files

Files:         632 new,     0 changed,     0 unmodified
Dirs:           22 new,     0 changed,     0 unmodified
Added to the repo: 11.377 MiB

processed 632 files, 11.151 MiB in 0:00
snapshot 20b821a0 saved
~> mv ~/Desktop/zz ~/devel/ # move the repo
~> export RESTIC_REPOSITORY=/home/gurkan/devel/zz # update env
~> cd Downloads
~> restic snapshots
repository 9953871b opened successfully, password is correct
ID        Time                 Host        Tags        Paths
---------------------------------------------------------------------------
20b821a0  2022-01-15 10:26:52  innodellix              /home/gurkan/Zomboid
---------------------------------------------------------------------------
1 snapshots
~> restic restore -t . --path ./Zomboid 20b821a0 # restore with relative
repository 9953871b opened successfully, password is correct
restoring <Snapshot 20b821a0 of [/home/gurkan/Zomboid] at 2022-01-15 10:26:52.316193642 +0000 UTC by gurkan@innodellix> to .

I think the difference here is that I am specifying latest for snapshot and relying on the path to resolve the backup to be restored. In your example you are providing an explicit snapshot id.

I think thst it may be difficult to make restic behave like tar. Usually with tar we don’t incrementally append files to a tar archive. However tar has an append option. tar never stores the absolute path to the files. But restic seems to care about the absolute path, though the user doesn’t specify it.

At this stage I think it may not be easy to get the same kind of behavior as in tar with restic without breaking changes.

I faced this issue when I was trying to restore a backup where my home directory was named differently in my new machine.

Thanks for taking a look.

@senthilbaboo You never answered this question by @gurkan. Please do (show the output of the snapshots command)!

Sorry about not being clear there. Restic shows the absolute path when I say to list snapshots. That was the whole point about my last comment. Restic does not seem to store only the relative path. Even if the user had specified only the relative path, restic is trying to resolve the absolute path and that is why I mentioned that it is inconsistent with tools like tar and I also think it won’t be easy to change this behavior now.

Yeah. This is expected. When you give path arguments e.g. to restore's --include you are matching on the actual path shown in the snapshot. Perhaps a bit surprising but now you know :slight_smile:

Yeah, I get that. My point was that it was not consistent with other tools like tar. This may bring up some issues when the source directory structure changes when restoring backups (which happened in my case).

In my case, I have ansible scripts which sets up new machines and also restores backups. I just changed home directory structure in the new machine. And the setup started failing.

In my humble opinion, when a user specifies a relative path during backup, the restore should work with the relative path. But it doesn’t.

I ended up doing a few things manually. I have to think about a better a way to automate to overcome the issue here.

Feel free to try out this PR if you wish: backup: Add options --set-path and --set-paths-from by aawsome · Pull Request #3200 · restic/restic · GitHub

Sure. Thanks! Let me try.