Local backup problem from non-bootable driver to external non-botable driver in desktop PC

Pelt · August 29, 2021, 12:57pm

Yesterday I started my testing of restic 0.12.1 on Manjaro laptop (with one bootalble SSD driver). Here both source data (in folders and files) and restic-repo are within the same /home main directory, but under different sub-directories of /home. All OK (repo init, backup, listing, restoration, forget etc…).

Based on this I have bee trying to make restic (0.12.1) backup on Manjaro desktop PC from directory (called here below: A. Works) located in mountable/unmountable non-bootable mass storage device (normal HDD called here below: 500Gt) to repo in external SATA hard drive (called here below: My10). All mentioned drivers have been mounted before the following commands and path(s) used here in backup commands are correct.

My 1st type commands were:

restic -r /run/media/user/My10/srv/restic-repo -–verbose backup /run/media/user/500Gt/’A. Works’

and the same with / in the end => they both give the same negative result, i.e.:

’’…/run/media/user/500Gt/’A. does not exist, skipping
Works’ does not exist, skipping
Fatal: all target directories/files do not exist’’

Then I thought that maybe restic cannot find the source directory because in this command both source and repo are running via …/run/media/user/ directory.

2.) Therefore to mitigate this situation I started Timeshift and launched the 2nd type commands to run only via …/run/ – directory, i.e. one single point of contact between these commands:

restic -r /run/media/user/My10/srv/restic-repo -–verbose backup /run/timeshift/backup/500Gt/’A. Works’

and the same with / in the end => they both give the same negative result as with the 1st type commands above.

I’m asking for advice how to avoid this difficulty in finding correctly source directory for successful running the backup command with this kind of task.

Nev · August 29, 2021, 1:53pm

Hi Pelt,
It doesn’t matter if both the source and repo are subdirectories in /run/media/user/, so forget your point (2) above. Your error message is telling you:

That your required source ("/run/media/user/500Gt/A. Works") is being understood as two separate sources ("/run/media/user/500Gt/'A." and "Works'"), and
That neither "/run/media/user/500Gt/'A." nor "Works'" has been found.

This doesn’t seem to be a restic thing: your single quotes are being treated by the shell as part of the argument, and thus your whitespace is seen as a separator between arguments, rather than as part of the single path. It’s not immediately clear to me why that is: what you’ve written above looks like it should work, if run as typed. Thoughts:

Have you tried different ways to escape the space, such as /run/media/user/500Gt/A.\ Works or "/run/media/user/500Gt/A. Works"
Are you running the command in some other way than just typing it on the command line? Are you using some other symbol rather than the standard single quotes?

Whatever it is, something’s messing with the shell’s interpretation of your quotes.

764287 · August 29, 2021, 1:57pm

’ is not the same as '.

Pelt · August 29, 2021, 4:54pm

@Nev Thank you for your reply and suggestions ! I’ll have possibility to try them tomorrow pm CET and let you know immediately about the results.

I can reply to item 2 of your thoughts as follows:

2.1) I’ve been running the commands exactly as they have been written in my opening mail. Btw, this same way of writing commands works OK, for instance, with my rsync and well as many other terminal operated programs I’ve been using. In restic Manual I did not notice that commands should be written in some other way. Or, evidently, I do not understand correctly your question.

2.2 ) In commands I’ve been using only those symbols which I wrote in my original opening mail, including single quotes.

Hopefully after tomorrow’s tests I can report some positive news.

Pelt · August 30, 2021, 2:56am

@Nev I made new tests and my findings are:

1.0) In this verison (0.12.1) of restic the name of the source file or source directory must be written either:
1.1.) without empty space(s) in the middle of the name (like: A_Works), or
1.2.) using double quote marks before and after the name with empty space in the middle of the name (like “B. Addresses”); single quote marks will lead to fatal failure in reading the source directory name.

2.) Commands ending
restic -r /run/media/user/My10/srv1/restic-repo2 --verbose backup /run/media/user/500Gt/“B. Addresses”
or
restic -r /run/media/user/My10/srv1/restic-repo2 --verbose backup /run/media/user/500Gt/“B. Addresses”/

are both correct, i.e. after source directory name (here “B. Addresses”) can be without or with / mark, both ways give the same successful snapshots. Listing of 4 successful test snapshots made according to my findings 1.1, 1.2 and 2. is here below:

   ~  restic -r /run/media/user/My10/srv1/restic-repo2 snapshots
enter password for repository: **
repository 856e07cf opened successfully, password is correct
ID Time Host Tags Paths
------------------------------------------------------------------------------------------------------
b531001f 2021-08-30 … hp … /run/media/user/500Gt/A_Works
27dbfc29 2021-08-30 … hp … /run/media/user/500Gt/B_Addresses
feb4b3ee 2021-08-30 … hp … /run/media/user/500Gt/B. Addresses
4838c330 2021-08-30 … hp … /run/media/user/500Gt/B. Addresses
------------------------------------------------------------------------------------------------------
4 snapshots
**    ~ 

I guess you might have some contacts with restic Manual editors. Could you to propose to them to take a look at these findings in 1.1) and 1.2) and - if possible - edit the Manual accordingly (in case restic developers do not want to make single quote marks accepted).

764287 · August 30, 2021, 7:52am

Please note that there are 2 different types of quotation marks in english: straight (' and ") and curved (‘ ’ and “ ”) (see Wikipedia for more information). You are using curved quotation marks for all of your commands. I have no idea why double curved quotation marks work for you because neither single nor double curved quotation marks work for me.

user:~$ restic --quiet backup /home/user/'B. Addresses'

user:~$ restic --quiet backup /home/user/"B. Addresses"

user:~$ restic --quiet backup /home/user/’B. Addresses’
/home/user/’B. does not exist, skipping
Addresses’ does not exist, skipping
Fatal: all target directories/files do not exist

user:~$ restic --quiet backup /home/user/“B. Addresses”
/home/user/“B. does not exist, skipping
Addresses” does not exist, skipping
Fatal: all target directories/files do not exist

user:~$ restic snapshots 
repository 69290d6c opened successfully, password is correct
ID        Time                 Host                   Tags        Paths
-----------------------------------------------------------------------------------------
020d61d2  2021-08-30 09:40:57  test.system                        /home/user/B. Addresses
f59e1e9b  2021-08-30 09:41:02  test.system                        /home/user/B. Addresses
-----------------------------------------------------------------------------------------
2 snapshots

Which language and keymap do you use on your system? I’m wondering if that makes a difference.

Nev · August 30, 2021, 7:53am

Hi @Pelt, as @764287 succinctly explained, looks like you might have been using a non-standard single quote - I just tried it, and certainly a normal single quote worked for me as expected. Glad to hear you’ve got it working now!

Pelt · August 30, 2021, 2:33pm

@Nev Thank you very much for your replies and test results.

I try to reply to your questions as follows, I’m sorry for long answer:

1.) In this Manjaro 21.1 Gnome installation the keyboard layout, please see below;
This command [ nano /etc/default/keyboard ] gives this info about the keyboard layout:
# KEYBOARD CONFIGURATION FILE
# Consult the keyboard(5) manual page.
XKBMODEL=“pc105”
XKBLAYOUT=“fi”
XKBVARIANT=“nodeadkeys”
XKBOPTIONS=""
BACKSPACE=“guess”

Command [ localectl status ] gives this info about the keyboard layout:
System Locale: LANG=fi_FI.UTF-8
VC Keymap: fi
X11 Layout: fi
X11 Variant: nodeadkeys

Keyboard model installed from Manjaro OS installer is Generic 105-key using Finnish classic keyboard layout without empty places (and it includes also these 3 letters used in Finnish and Swedish languages: ä = a umlaut and ö= o umlaut, å = so called ’swedish O’). When running of rsync of cp or other terminal controlled applications I have used straight quotation marks ( ’ and ’’ ) which to - my understanding - are the same as shown in box ’Finnish / Standard’ in Wikipedia article you referred to. In my previous restic testings I used these same straight quotation marks. Enclosed snapshot with text showing the place of of the key for single quotation mark.

2.) Now comes the interesting part re correct name of source directory:

After my today’s 1st reply I started to run restic commands with real work directories using the findings of my previous reply about double quotation marks. However, for some reason, it came out now, that double quotation marks are not accepted, instead single quotation marks as well as one space in the middle of the source directory name is accepted, i.e. now the the same result as you got also, i.e:

   /  restic -r /run/media/user/My10/srv1/restic-repo1 backup /run/media/user/500Gt/‘A. Works’
enter password for repository:
repository d5923059 opened successfully, password is correct
no parent snapshot found, will read all files

Files: 4426 new, 0 changed, 0 unmodified
Dirs: 925 new, 0 changed, 0 unmodified
Added to the repo: 2.084 GiB

processed 4426 files, 2.980 GiB in 3:09
snapshot 1b84aaba saved
   /    3m 49s 

Note: In this OS when quotation marks around source directory name are given correctly in the command line, the directory name in question becomes yellow including the quotation marks.

It came to mind that, it could be very handy to have possibility for DRY RUN in restic before executing the actual backup command run, i.e. similar to -n flag in the rsync -aunv command. Then it would be easy to edit your command line to become correct and executable. I did not find DRY RUN flag in Manual.

(FYI only: This original backup of almost 3 Gb was was created in 3m49s; original backup of the same directory with rsync -auv command it took 2m7s and and with the cp -a command it took 1m3s.)

Mounting / u-mounting of this repo1 to / from /tmp/restic-repo was also successful:

   /tmp  cd … 
   /  restic -r /run/media/user/My10/srv1/restic-repo1 mount /tmp/restic-mount
enter password for repository:
repository d5923059 opened successfully, password is correct
Now serving the repository at /tmp/restic-mount
When finished, quit with Ctrl-c or umount the mountpoint.
signal interrupt received, cleaning up
   /   INT ✘  1h 2m 59s 

So, wrapping up: I shall use on single quotation marks and space in the source directory names. In case the command says something about missing directory, I shall not copy-paste the previous command, but delete previous quotations marks and insert manually new ones in terminal. In case problems shall appear, I’ll try come back to this question.