I created a local restic-repo on /media/kdv/LOCALBACKUP/
I did a backup of my root directory snap ( with exclusions ) using that repo. ( /mnt/rootsnap/ )
I can use “restic snapshots” and it lists the snap.
I can use “du” to see that the snap contains 85G of data.
However:
when I run “restic restore snapshotid --target /mnt/spare/” ,
In the target directory, there is “mnt/”, and in mnt/ there is “rootsnap/”
But there is nothing restored under mnt/rootsnap
What am I doing wrong?
Thanks
Based on your description it sounds as if no files were backed up. Perhaps you excluded too much or something. Or you are not looking in the target dir correctly.
Please run restic -r /media/kdv/LOCALBACKUP ls <snapshotid> and investigate the output. This commands lists which files are included in the snapshot you provide as <snapshotid>.
Hi. Many thanks for taking the trouble to respond.
Firstly, I wasn’t clear, /media/kdv/LOCALBACKUP/ was the containing directory, the repo is /media/kdv/LOCALBACKUP/restic-kdv
I use the RESTIC_REPOSITORY and RESTIC_PASSWORD_FILE environmental variables at all times so I don’t have to keep typing ( and possibly mistyping! ) them each time I issue a restic command.
I excluded the root directories which shouldn’t be backed up ( such as /proc, /run, etc ), but still many should have been backed up, such as /bin/, /usr, /etc.
And the same problem occurs with a snapshot I make at a different directory with a command with no excludes at all, i.e. restic backup -x -v /mnt/vboxsnap
If I perform : du -sh /media/kdv/LOCALBACKUP/restic-kdv/data
I get a count of about 85G.
And the command: find /media/kdv/LOCALBACKUP/restic-kdv/data | wc
gives me a count of something more than 18000 files.
So something is being put there.
But, you’re right, restic ls does not list any files, just the top level directories.
So restic seems to be somehow forgetting the data it has backed up. Am I wrong?
Best regards.
I never ever heard of restic forgetting what’s in its repository. You could run restic check to verify that the integrity of the data seems to be alright, just to verify.
Other than that, you should probably start listing all relevant information, such as the exact script/commands you use to back up, including any excludes.
Are you backing up Btrfs snapshots? And maybe with --one-file-system? Btrfs snapshots are considered different filesystems, in this case restic will not do what you expected and store only the mountpoint. It happened to me once and I wondered why the backup is so slim 
Thanks, rawtaz. Again I appreciate your time and trouble.
The output of “restic check” is as follows:
The script use for backup is:
__
/usr/local/bin/mountrootsnap
restic backup -x -v --exclude-file=/root/Documents/restic.exclude /mnt/rootsnap
/usr/local/bin/delrootsnap
__
/usr/local/bin/mountrootsnap is as follows:
__
mkdir -p /mnt/rootsnap
lvcreate -s -L 2G -n rootsnap /dev/vgroot/root >/dev/null
mount /dev/vgroot/rootsnap /mnt/rootsnap
__
/root/Documents/restic-exclude is:
__
/mnt/rootsnap/crypt/*
/mnt/rootsnap/home/*
/mnt/rootsnap/imports/*
/mnt/rootsnap/lost+found
/mnt/rootsnap/media/*
/mnt/rootsnap/mnt/*
/mnt/rootsnap/opendrive/*
/mnt/rootsnap/proc/*
/mnt/rootsnap/run/*
/mnt/rootsnap/snap/*
/mnt/rootsnap/srv/*
/mnt/rootsnap/store/*
/mnt/rootsnap/tmp/*
/mnt/rootsnap/var/tmp/*
/mnt/rootsnap/root/.cache/*
__
Not sure how helpful that is, but I’m hoping. I think greater minds than mine are needed!
Regards
Hi sniner. Thanks for your thoughts.
It’s not btrfs snapshot, it’s lvm snapshot, but the same principle applies I guess.
/mnt/rootsnap is a separate filesystem, it’s in its own logical volume.
But I’m not convinced I have a slim backup … as I said earlier, it’s 85G and 18K files fat!
sniner, you have given me the needed kick.
I tried another snapshot, using restic without the -x. And it worked, I was able to restore the backed up files. I’m still not sure why this is the case, as the backup was only within the snapshot.
But it is what it is. You pointed me in the right direction.
Thank you to both you and rawtaz!
Why did you use -x in the first place, when you don’t even seem to understand its implications?
Glad it worked out. I’ll mark @sniner’s post as the solution.
FWIW, this thread is another good example of why one should provide the complete information such as what commands one run, right off the bat. Had this been done, we could have immediately spotted the use of -x and mounts etc, and let you know without much debugging.
Ok. I’ll consider myself slapped down.
Don’t worry, I won’t trouble you here again.
Sorry, that wasn’t meant as slapping, it was actually a pretty genuine question 
I don’t understand how come you managed to find that flag in the first place, because it’s a pretty specialized flag, so I was surprised when it was the culpruit. Didn’t mean to be rude. You haven’t troubled anyone at all.
Ok, thanks for clarifying.
I’m happy to address the points you made.
Firstly, I generally find that when asking a question about OS or software issues, it’s best to keep the initial enquiry short. Very long questions, in my experience, are less likely to be read or responded to. Then, if someone is generous enough with their time to try to help, they might suggest further needed information, which you did, and which I provided
With regard to the -x ( --one-file-system ) parameter, I do indeed know what that means. It is in the restic-backup man page. With new software, I really do RTFM.
I used it because I often mount volumes (e.g. usb memory sticks) within my / or /home directories. When a snapshot is made, it carries over those mountpoints to the snap. I don’t want mounted volumes to be included in the backup, hence the “-x”.
Yes, I could use exclude commands. But with things like usb sticks, they come and go, and I don’t really want to remember to change the exclude file for restic each time I change them.
Furthermore, I think restic’s behaviour is odd. If I create a snapshot, then copy it with “rsync -ax”, where the -x in rsync is described just the same as it is in restic, all behaves as I would have expected. /mnt/homesnap/ and all its contents are copied exactly, without the mounted volumes. I assumed (wrongly) that restic would behave similarly.
And yet. As I said in earlier posts, when I examined the restic repository, the size would indicate that my desired files were actually being backed up. So did the time it took to perform the backup. Restic seemed to be backing up the way I expected, but neither “restic ls” nor “restic restore” was able to see beyond the base directory. I’d go so far as to think that’s more like a bug than a feature, but of course I’m not a restic guru, so there may be a valid reason for that behaviour. If so, it probably needs a few more clues in the man page.
I hope that clarifies my thinking. I’m still wondering how to exclude mounted volumes from the backups without constantly editing the exclude files.
Regards
I think we should investigate that. I haven’t seen any previous reports of restic not backing up the files within the -x filesystem and then be able to restore them properly, so I don’t think there’s a bug around it. But given what you said it needs to be tested again just to be sure.
I agree that if you have 85 GB of data in your data folder, there was definitely something backed up. One possible cause of not being able to restore anything is if that data belongs to snapshots you since deleted and then never backed that data up again. But it’s hard to tell.
Can you meanwhile post the command and output of restic snapshots? If there’s some specific folder names you want to obfuscate feel free to do so, as long as it’s done consistently in that post and future posts.
Happy to show you the snapshots.
27 snapshots
Regards
Okay. I thought this one had been put to bed.
The story up to now: I was running restic with an -x parameter, and it only backed up the top level directory.
It was suggested here that the -x might be the problem, so I ran it without and it worked. Problem solved, I thought.
No. The problem is not related to the “-x”.
Originally, I was calling my script called “backup” ( I’ll reveal it below ) automatically through a cron job. When I tried it without the -x, I ran “backup” manually from a command line, exactly as I was calling it from the cron job.
On further examination, it seems that restic functions properly when backup is called from the command line, and fails when it’s called by cron. *With our without the -x parameter".
So now my poor befuddled brain is even more confused.
Here is the “backup” script I am using.
#!/bin/bash
export RESTIC_PASSWORD_FILE=/usr/local/etc/restic-pw
export RESTIC_REPOSITORY=/dummy
case "$1" in
local )
export RESTIC_REPOSITORY=/media/kdv/LOCALBACKUP/restic-kdv
;;
jotta )
export RESTIC_REPOSITORY=rclone:jotta:Backups/restic-kdv
;;
esac
if ! restic snapshots >/dev/null 2>&1 ; then exit 1 ; fi
/usr/local/bin/mountrootsnap
restic backup -v --exclude-file=/root/Documents/restic.exclude /mnt/rootsnap
/usr/local/bin/delrootsnap
/usr/local/bin/mounthomesnap
restic backup -v --exclude-file=/home/restic-exclude /mnt/homesnap
/usr/local/bin/delhomesnap
/usr/local/bin/mountvboxsnap
restic backup -v /mnt/vboxsnap
/usr/local/bin/delvboxsnap
restic forget --keep-daily 7 --keep-weekly 3 --keep-monthly 3
If I understand correctly you are mounting a directory, backup the directory and unmount the directory in your script. A possible reason why the restore is empty is that mounting the directory isn’t working (as expected). Can you comment/remove the unmounting part in your script, run it and afterwards check if the mount points contains data and looks like expected?
Yes, certainly worth a try. I’ll do that, and report back. Thanks for your thoughts.
You’re right. The snapshots aren’t mounting when called from within a cron job!
Apologies to the developers of restic. No fault there at all.
Now, I have to work out why.
This is a good reason to use && when chaining commands that depend on prior commands to complete successfully. Assuming that your various snap scripts return non-zero exit codes on failure, you could do something like this:
/usr/local/bin/mountrootsnap && \
restic backup -v --exclude-file=/root/Documents/restic.exclude /mnt/rootsnap && \
/usr/local/bin/delrootsnap
/usr/local/bin/mounthomesnap && \
restic backup -v --exclude-file=/home/restic-exclude /mnt/homesnap && \
/usr/local/bin/delhomesnap
# And so on
This way you won’t wind up with an empty backup if mounting of a snapshot fails – you’ll end up with no backup at all which is a much better indication of where things went wrong (before restic was invoked) and ensures that your forget policy doesn’t throw out older, good backups to retain newer, empty backups.
Yes, very good advice, I will implement something along those lines. I admit I didn’t put enough thought into it, and I paid the price.
I found out why the mount wasn’t happening. The crontab file needed a PATH= directory in it. When called manually, my backup script inherited everything it needed, but not when called from cron.