How to: Retrieve A File From a Snapshot

powerhouse · July 21, 2020, 6:23am

Is it possible to retrieve a file in a previous snapshot?
Specifically, config.php in /usr/share/squirrelmail/config then override the current config.php file in /usr/share/squirrelmail/config

If so how?

Thanks, so happy I found this community as well!

cdhowie · July 21, 2020, 4:16pm

Sure, just use:

restic restore --target=/ --include=/usr/share/squirrelmail/config/config.php $SNAPSHOT

Replacing $SNAPSHOT with the appropriate snapshot ID. Note that this restores the file in-place, which can be dangerous. If you want to ensure that the file is what you expected first, you can restore to a temporary location first, then move the file.

mkdir /tmp/restore
restic restore --target=/tmp/restore --include=/usr/share/squirrelmail/config/config.php $SNAPSHOT

Then inspect /tmp/restore to make sure it contains the file you expect. If it does, move it over the file you want to replace.

alexweiss · July 21, 2020, 5:35pm

Another common method if it is just about a single file is to use restic mount (on supported platforms like linux): This “mounts” all contents of your repository to a given path.
This can then be navigated by your favorite file browser (allows all read only file operations, e.g. reading the content or diffing with other files) and restoring works by just copying the correctly identified file to its original location.

If you restore big amounts of data, the restic restore command should be preferred, however.

powerhouse · July 21, 2020, 7:10pm

Awesome, thanks so much. I’m leveling up here.
I keep getting Fatal: Please specify repository location (-r)

sudo restic restore --target=/mnt/raid5 --include=/usr/share/squirrelmail/config/config.php b55535fc
Fatal: Please specify repository location (-r)

sudo mkdir /tmp/restore
sudo restic restore --target=/tmp/restore --include=/usr/share/squirrelmail/config/config.php b55535fc
Fatal: Please specify repository location (-r)

cdhowie · July 21, 2020, 7:26pm

I tried using this in the past but found it to be horrendously slow on large repositories to the point where it was completely unusable. Maybe it works better on small repositories.

Well, yes, I assumed you would be specifying this as well. I have scripts that set RESTIC_REPOSITORY so I don’t need -r. If you need to specify the location then just add that parameter.

alexweiss · July 21, 2020, 8:25pm

Recently, Improvements on speed in general and mount in particular have been done and some more are on the way. You might want to try out the latest beta (which should be already faster especially for large files within your repo) or this optimization:

github.com/restic/restic

Make `restic mount` faster and consume less memory

restic:master ← aawsome:optimize-fuse

opened 04:29PM - 18 Feb 20 UTC

aawsome

+108 -66

What is the purpose of this change? What does it change? ----------------------…---------------------------------- In the FUSE implementation restic data structure have been read (meaning loading blobs from the repository) and internal data structures have been created when a FUSE "node" was created. This was the case, for instance, for all item within a dir when this dir was accessed. This lead to quite some memory usage and made the FUSE filesystem "unresponsive". This PR changes this behavior and only reads restic data structure when the information is really needed. Moreover, the internal data structure have been optimized. Also concurrent operation within FUSE should now work correctly. (FUSE documentation says that access should be synced which wasn't) Was the change discussed in an issue or in the forum before? ------------------------------------------------------------ closes #1680 Thanks @greatroar for issuing #2585 which makes me think about whether the bottleneck really is only reading tree blobs. Checklist --------- - [X] I have read the [Contribution Guidelines](https://github.com/restic/restic/blob/master/CONTRIBUTING.md#providing-patches) - I have not added tests for all changes in this PR - I have not added documentation for the changes (in the manual) - [X] There's a new file in `changelog/unreleased/` that describes the changes for our users (template [here](https://github.com/restic/restic/blob/master/changelog/TEMPLATE)) - [X] I have run `gofmt` on the code in all commits - [X] All commit messages are formatted in the same style as [the other commits in the repo](https://github.com/restic/restic/blob/master/CONTRIBUTING.md#git-commits) - [X] I'm done, this Pull Request is ready for review

(still in progress, however)

powerhouse · July 21, 2020, 9:49pm

I see, the computer is telling me it needs the parameter -r which is in the manual. Is a parameter and a flag the same thing in this case?

yum -v repolist | egrep restic
Gives me

Repo-filename: /etc/yum.repos.d/_copr_copart-restic.repo

So, location is /etc/yum.repos.d/_corp_copart-restic.repo

restic restore --target=/tmp/restore --include=/usr/share/squirrelmail/config/config.php b55535fc --repo “/etc/yum.repos.d/_copr_copart-restic.repo”

Fatal: unable to open repo at /etc/yum.repos.d/_copr_copart-restic.repo: ReadDir: open /etc/yum.repos.d/_copr_copart-restic.repo/keys: not a directory
Okay, just select the dir not the actual repo.

sudo restic restore --target=/tmp/restore --include=/usr/share/squirrelmail/config/config.php b55535fc --repo “/etc/yum.repos.d”
Fatal: unable to open config file: Stat: stat /etc/yum.repos.d*/config*: no such file or directory

hmm…I see the file but the computer is telling me there is no file.

Is there a repository at the following location?
/etc/yum.repos.d

I see the /config getting added in there. Possible issue.

/etc/yum.repos.d shows the repo.

What am I missing?

rawtaz · July 21, 2020, 10:23pm

Yes.

This has nothing to do with the restic repository, which is the location what you backed up your files to.

Here is the documentation that talks about creating a repository. When you set up your backups, what did you use as repository then? Or have you not started using restic yet?

Please read the manual I linked to above if you haven’t already

powerhouse · July 21, 2020, 11:29pm

This has nothing to do with the restic repository, which is the location what you backed up your files to.

Oh okay, I originally tried that but had the quotations in there so that was a misfire.

This goes through successfully

restic restore --target=/tmp/restore --include=/usr/share/squirrelmail/config/config.php b55535fc --repo /run/media/orca/DataCabinet

repository e72f1d68 opened successfully, password is correct
found 2 old cache directories in /root/.cache/restic, run restic cache --cleanup to remove them
restoring <Snapshot b55535fc of [/home/orca] at 2020-07-20 11:29:09.163974804 -0700 PDT by root@orcacomputers.orcainbox> to /tmp/restore

Restore directory is empty =(
The file should be in there no?

/tmp/restore
[orca@orcacomputers restore]$ ll
total 0

Or have you not started using restic yet?

Been doing backups to my external hard drive for a while now.

cdhowie · July 22, 2020, 2:35am

I have no idea what is in your backups. Try restic ls b55535fc | less to show the contents of the backup. Run man less if you aren’t sure how to use the less pager (it will let you scroll through the output, which is useful if there are hundreds of files in the backup).

powerhouse · July 22, 2020, 5:50am

ls restic /run/media/orca/DataCabinet/snapshots b55535fc | less

snapshots
Just shows the snaps, not the files.

restic restore --target=/tmp/restore --include=/usr/share/squirrelmail/config/config_default.php b55535fc --repo /run/media/orca/DataCabinet

restoring <Snapshot b55535fc of [/home/orca] at 2020-07-20 11:29:09.163974804 -0700 PDT by root@orcacomputers.orcainbox> to /tmp/restore
Says it went through but no file in directory.

So, must see in the snapshot to confirm I have the file directory correct. File was incorrect, I needed config_default.php, not config.php, but the same issue happened. File didn’t go to /tmp/restore.

The less pager comes up but it only shows the snaps, it’s not accessing the contents of b55535fc snapshot.

man less
wow, lots of options there!

cdhowie · July 22, 2020, 5:51am

That’s not the same command as the one in my post. You have transposed restic and ls, and you’ve forgotten -r.

powerhouse · July 22, 2020, 2:37pm

shoot, I tried more commands unsuccessfully. Each command says it went through but the /tmp/restore is still empty as per

ll -al

I tried these commands this morning.

restic restore --target=/tmp/restore --include=/usr/share/squirrelmail/config/config_default.php b55535fc --repo /run/media/orca/DataCabinet

restic restore --target=/tmp/restore --include=/usr/share/squirrelmail/config/config_default.php b55535fc --repo=/run/media/orca/DataCabinet

restic restore --target=/tmp/restore --include=/usr/share/squirrelmail/config/config_default.php b55535fc -r /run/media/orca/DataCabinet

restic restore --target=/tmp/restore --include=/usr/share/squirrelmail/config/config_default.php b55535fc --repo=/run/media/orca/DataCabinet

restic restore --target=/tmp/restore --include=/usr/share/squirrelmail/config/config_default.php b55535fc -r /run/media/orca/DataCabinet | less

cdhowie · July 22, 2020, 6:42pm

Show the output of:

restic ls -l -r /run/media/orca/DataCabinet b55535fc | grep 'squirrelmail/config'

powerhouse · July 22, 2020, 6:56pm

restic ls -l -r /run/media/orca/DataCabinet b55535fc | grep ‘squirrelmail/config’
Fatal: unable to open repo at /run/media/orca/DataCabinet: ReadDir: open /run/media/orca/DataCabinet/keys: permission denied

sudo restic ls -l -r /run/media/orca/DataCabinet b55535fc | grep ‘squirrelmail/config’
[sudo] password for orca:
enter password for repository:
[orca@orcacomputers ~]$

result = empty = 0

thanks for the help cdhowie

cdhowie · July 22, 2020, 7:03pm

So it looks like there is no such file in the backup. If you run it without the grep pipeline, does it dump a list of files? Do you recognize any of them?

powerhouse · July 22, 2020, 8:02pm

restic ls -l -r /run/media/orca/DataCabinet/ b55535fc

outputs lots of lines which I recognize so that’s good.

sudo restic ls -l -r /run/media/orca/DataCabinet/ b55535fc | egrep ‘config.php’
Outputs files, but not the file I am looking for, Media Wiki and phpBB only.

The actual file I messed up is /usr/share/squirrelmail/config/config_default.php so that is what I should be searching for.

restic ls -l -r /run/media/orca/DataCabinet/ b55535fc | egrep ‘config_default.php’

returns NIL which I don’t understand, like you said “the file is not there” but it has to be I back up my entire filesystem with:

restic -r /run/media/orca/DataCabinet backup ~

The snapshots are about 40GB so this tells me the entire system is backed up so the file must be there unless my backup command is wrong.

Is there a way I can see the file directory of the snapshots?

rawtaz · July 22, 2020, 10:21pm

This is a file in the directory with path /usr/share/squirrelmail/config/. This is completely outside of your home directory (~), assuming your home directory isn’t somewhere in that path, which it most likely isn’t.

This backs up only your home directory, not e.g. /usr/share/squirrelmail/config/. Usually your home directory is something like /home/<username>, mine is /home/rawtaz.

This means that you have never backed up the file you are looking for. Please verify this by showing us the output of echo $HOME. If the beginning of the path that this command outputs doesn’t match /usr/share/squirrelmail/config/ then you simply never backed this file up in the first place, and that is why it is not in your backup and that is why it can obviously not be restored from the backup.

powerhouse · July 22, 2020, 11:49pm

This is completely outside of your home directory ( ~ )

Bingo! man good eyes, rawtaz!

[orca@orcacomputers ~]$ echo $HOME
/home/orca

restic -r /run/media/orca/DataCabinet backup ~

Can I just add the / on the end of ~ so:

restic -r /run/media/orca/DataCabinet backup ~ /

I want to back up everything at once. ~ and /

This is a blessing in disguise actually. Here I was with total piece of mind knowing my company is now safe from data loss.

rawtaz · July 22, 2020, 11:59pm

Honestly, and I say this not to be rude but to make you understand - you totally need to educate yourself on the basics of using your operating system. You seem very lost in what even the most basic commands do or how to structure them, what the files in your filesystems are, what the different paths in your filesystem are, and which paths cover what (for example, / will cover ~ so if you back up / you don’t need to also back up ~)

So yes, the answer to your question is that you can just add / at the end of your backup command, but if you want to back up everything you can just replace the ~ with / instead since that’s the more proper representation of “everything”.

Note that you’ll probably want to exclude certain directories, there’s a lot of files that aren’t relevant to back up at all, e.g. some runtime and system files. There should be a bunch of topics about backing up (and I think also restoring) full Linux filesystems or similar, feel free to search a bit to perhaps find them. There’s a lot of people backing up their entire system.

The blessing in disguise isn’t so much due to an actual disguise, you should know that if you back up ~ this will not include /. There’s nothing disguised here at all

Apologies if you feel offended, but I’m just bringing up the elephant in the room. Hopefully for the best - if you continue like this it’s just a matter of time before you run into some really big problem or data loss or vulnerability due to not knowing how your system(s) work. Noone wants that to happen!