How to: Retrieve A File From a Snapshot

Awesome, thanks so much. I’m leveling up here.
I keep getting Fatal: Please specify repository location (-r)

sudo restic restore --target=/mnt/raid5 --include=/usr/share/squirrelmail/config/config.php b55535fc
Fatal: Please specify repository location (-r)

sudo mkdir /tmp/restore
sudo restic restore --target=/tmp/restore --include=/usr/share/squirrelmail/config/config.php b55535fc
Fatal: Please specify repository location (-r)

I tried using this in the past but found it to be horrendously slow on large repositories to the point where it was completely unusable. Maybe it works better on small repositories.

Well, yes, I assumed you would be specifying this as well. I have scripts that set RESTIC_REPOSITORY so I don’t need -r. If you need to specify the location then just add that parameter.

2 Likes

Recently, Improvements on speed in general and mount in particular have been done and some more are on the way. You might want to try out the latest beta (which should be already faster especially for large files within your repo) or this optimization:


(still in progress, however)
1 Like

I see, the computer is telling me it needs the parameter -r which is in the manual. Is a parameter and a flag the same thing in this case?

yum -v repolist | egrep restic
Gives me

Repo-filename: /etc/yum.repos.d/_copr_copart-restic.repo

So, location is /etc/yum.repos.d/_corp_copart-restic.repo

restic restore --target=/tmp/restore --include=/usr/share/squirrelmail/config/config.php b55535fc --repo “/etc/yum.repos.d/_copr_copart-restic.repo”

Fatal: unable to open repo at /etc/yum.repos.d/_copr_copart-restic.repo: ReadDir: open /etc/yum.repos.d/_copr_copart-restic.repo/keys: not a directory
Okay, just select the dir not the actual repo.

sudo restic restore --target=/tmp/restore --include=/usr/share/squirrelmail/config/config.php b55535fc --repo “/etc/yum.repos.d”
Fatal: unable to open config file: Stat: stat /etc/yum.repos.d*/config*: no such file or directory

hmm…I see the file but the computer is telling me there is no file.

Is there a repository at the following location?
/etc/yum.repos.d

I see the /config getting added in there. Possible issue.

/etc/yum.repos.d shows the repo.

What am I missing?

Yes.

This has nothing to do with the restic repository, which is the location what you backed up your files to.

Here is the documentation that talks about creating a repository. When you set up your backups, what did you use as repository then? Or have you not started using restic yet?

Please read the manual I linked to above if you haven’t already :slight_smile:

This has nothing to do with the restic repository, which is the location what you backed up your files to.

Oh okay, I originally tried that but had the quotations in there so that was a misfire.

This goes through successfully

restic restore --target=/tmp/restore --include=/usr/share/squirrelmail/config/config.php b55535fc --repo /run/media/orca/DataCabinet

repository e72f1d68 opened successfully, password is correct
found 2 old cache directories in /root/.cache/restic, run restic cache --cleanup to remove them
restoring <Snapshot b55535fc of [/home/orca] at 2020-07-20 11:29:09.163974804 -0700 PDT by root@orcacomputers.orcainbox> to /tmp/restore

Restore directory is empty =(
The file should be in there no?

/tmp/restore
[orca@orcacomputers restore]$ ll
total 0

Or have you not started using restic yet?

Been doing backups to my external hard drive for a while now.

I have no idea what is in your backups. Try restic ls b55535fc | less to show the contents of the backup. Run man less if you aren’t sure how to use the less pager (it will let you scroll through the output, which is useful if there are hundreds of files in the backup).

1 Like

ls restic /run/media/orca/DataCabinet/snapshots b55535fc | less

snapshots
Just shows the snaps, not the files.

restic restore --target=/tmp/restore --include=/usr/share/squirrelmail/config/config_default.php b55535fc --repo /run/media/orca/DataCabinet

restoring <Snapshot b55535fc of [/home/orca] at 2020-07-20 11:29:09.163974804 -0700 PDT by root@orcacomputers.orcainbox> to /tmp/restore
Says it went through but no file in directory.

So, must see in the snapshot to confirm I have the file directory correct. File was incorrect, I needed config_default.php, not config.php, but the same issue happened. File didn’t go to /tmp/restore.

The less pager comes up but it only shows the snaps, it’s not accessing the contents of b55535fc snapshot.

man less
wow, lots of options there!

That’s not the same command as the one in my post. You have transposed restic and ls, and you’ve forgotten -r.

2 Likes

shoot, I tried more commands unsuccessfully. Each command says it went through but the /tmp/restore is still empty as per

ll -al

I tried these commands this morning.

restic restore --target=/tmp/restore --include=/usr/share/squirrelmail/config/config_default.php b55535fc --repo /run/media/orca/DataCabinet

restic restore --target=/tmp/restore --include=/usr/share/squirrelmail/config/config_default.php b55535fc --repo=/run/media/orca/DataCabinet

restic restore --target=/tmp/restore --include=/usr/share/squirrelmail/config/config_default.php b55535fc -r /run/media/orca/DataCabinet

restic restore --target=/tmp/restore --include=/usr/share/squirrelmail/config/config_default.php b55535fc --repo=/run/media/orca/DataCabinet

restic restore --target=/tmp/restore --include=/usr/share/squirrelmail/config/config_default.php b55535fc -r /run/media/orca/DataCabinet | less

Show the output of:

restic ls -l -r /run/media/orca/DataCabinet b55535fc | grep 'squirrelmail/config'
1 Like

restic ls -l -r /run/media/orca/DataCabinet b55535fc | grep ‘squirrelmail/config’
Fatal: unable to open repo at /run/media/orca/DataCabinet: ReadDir: open /run/media/orca/DataCabinet/keys: permission denied

sudo restic ls -l -r /run/media/orca/DataCabinet b55535fc | grep ‘squirrelmail/config’
[sudo] password for orca:
enter password for repository:
[orca@orcacomputers ~]$

result = empty = 0

thanks for the help cdhowie

So it looks like there is no such file in the backup. If you run it without the grep pipeline, does it dump a list of files? Do you recognize any of them?

1 Like

restic ls -l -r /run/media/orca/DataCabinet/ b55535fc

outputs lots of lines which I recognize so that’s good.

sudo restic ls -l -r /run/media/orca/DataCabinet/ b55535fc | egrep ‘config.php’
Outputs files, but not the file I am looking for, Media Wiki and phpBB only.

The actual file I messed up is /usr/share/squirrelmail/config/config_default.php so that is what I should be searching for.

restic ls -l -r /run/media/orca/DataCabinet/ b55535fc | egrep ‘config_default.php’

returns NIL which I don’t understand, like you said “the file is not there” but it has to be I back up my entire filesystem with:

restic -r /run/media/orca/DataCabinet backup ~

The snapshots are about 40GB so this tells me the entire system is backed up so the file must be there unless my backup command is wrong.

Is there a way I can see the file directory of the snapshots?

This is a file in the directory with path /usr/share/squirrelmail/config/. This is completely outside of your home directory (~), assuming your home directory isn’t somewhere in that path, which it most likely isn’t.

This backs up only your home directory, not e.g. /usr/share/squirrelmail/config/. Usually your home directory is something like /home/<username>, mine is /home/rawtaz.

This means that you have never backed up the file you are looking for. Please verify this by showing us the output of echo $HOME. If the beginning of the path that this command outputs doesn’t match /usr/share/squirrelmail/config/ then you simply never backed this file up in the first place, and that is why it is not in your backup and that is why it can obviously not be restored from the backup.

2 Likes

This is completely outside of your home directory ( ~ )

Bingo! man good eyes, rawtaz!

[orca@orcacomputers ~]$ echo $HOME
/home/orca

restic -r /run/media/orca/DataCabinet backup ~

Can I just add the / on the end of ~ so:

restic -r /run/media/orca/DataCabinet backup ~ /

I want to back up everything at once. ~ and /

This is a blessing in disguise actually. Here I was with total piece of mind knowing my company is now safe from data loss.

Honestly, and I say this not to be rude but to make you understand - you totally need to educate yourself on the basics of using your operating system. You seem very lost in what even the most basic commands do or how to structure them, what the files in your filesystems are, what the different paths in your filesystem are, and which paths cover what (for example, / will cover ~ so if you back up / you don’t need to also back up ~)

So yes, the answer to your question is that you can just add / at the end of your backup command, but if you want to back up everything you can just replace the ~ with / instead since that’s the more proper representation of “everything”.

Note that you’ll probably want to exclude certain directories, there’s a lot of files that aren’t relevant to back up at all, e.g. some runtime and system files. There should be a bunch of topics about backing up (and I think also restoring) full Linux filesystems or similar, feel free to search a bit to perhaps find them. There’s a lot of people backing up their entire system.

The blessing in disguise isn’t so much due to an actual disguise, you should know that if you back up ~ this will not include /. There’s nothing disguised here at all :wink:

Apologies if you feel offended, but I’m just bringing up the elephant in the room. Hopefully for the best - if you continue like this it’s just a matter of time before you run into some really big problem or data loss or vulnerability due to not knowing how your system(s) work. Noone wants that to happen! :confused:

1 Like

~ = home
/ = root
Got it 100%

No worries, appreciate the father Crom in you.

you’ll probably want to exclude certain directories, there’s a lot of files that aren’t relevant to back up at all, e.g. some runtime and system files

10-4 I am seeing that now. ETA 17 hours to do a backup, total size 128 TiB, all 12 cores on my cpu are at capacity so I will stop it and back up my main software and /var/www for all webcontent. Then if I ever edit a configuration file or something I will back up just that file.

Thanks again, you helped me on my other thread a while back too. Much appreciated.

Note that you also probably want to add --one-file-system to your backup command.

Linux has a number of virtual filesystems that you DO NOT want to back up (such as /proc and /sys, and even /dev on modern distros). This calls back to what @rawtaz said about learning the basics of your system. You don’t actually want to back up everything visible from / since some of that stuff doesn’t actually exist on disk.

In particular, if you happen to have a removable USB drive mounted when the backup runs, the backup will also grab the contents of that drive.

--one-file-system prevents restic backup from “crossing filesystem boundaries” which means it will stay only within the filesystems explicitly listed as arguments. If / is the only storage volume in your system, then restic backup -r ... --one-file-system / will work. But if you have e.g. /home as a separate volume, then you would need to list both (/ /home) on the command line.

There is no one-size-fits-all answer here. You need to know your own system.

No, you don’t want to back up “just that file.” After restic backup has finished backing up this mountain of data, future runs will only back up chunks of files that changed and you should see follow-up backups complete very quickly.

If you back up only files that changed by hand, then you will not have a complete snapshot of everything anymore. If your disk suddenly dies and you want to restore everything from a backup, you’ll have to restore multiple backups in pieces to restore the system. Letting restic take complete snapshots (with its deduplication) should be pretty quick and gives you a single snapshot of the entire state of the system. (The snapshot will just share a lot of data with the prior snapshot.)

1 Like

Just for info: This is now included in the beta builds.
@cdhowie: Would you mind trying the latest beta with your large repositories?

IMO the restic mount command is very useful for various situations and I think it is especially a nice tool for people who are not that confident using the console…

1 Like