Usual & Exceptional use cases: Differing host & path names for same files

[blf]

Hello!

I’m new to this forum. I am currently evaluating various
backup & restore solutions for my equipment, and `restic’
is currently a leading contender. I am testing v0.9.6 on
Linux (Kubuntu & Lubuntu, various versions).

Whilst I’ve run across several potential issues, this is
only about one, one which is particularly bothering me.
First, however, a brief description of my plans/setup:

● Local backup only, from internal-HDD to one or more
removeable units (mainly HDDs); USB3 connection.

● Normal use case is running on/from the internal-HDD;
i.e., the internal-HDD is the root (and usually also
the boot device).

● Manually launched backup & restore.

● Exceptional use case is running on/from an external
unit, which may or may not be the external unit to
which the backup is being made (i.e., the one which
contains the restic-repository). This is one of my
contingency plans for a borked/suspect internal-HDD;
that is, to make an “emergency” backup before doing
much of anything else about the broked/suspect HDD.

● Each external unit with a root has (deliberately) a
different hostname. (This is a contingency in case
of a boot/root on different hardware; I don’t want
any host “spoofing” except by explicit decision.)

The restic v0.9.6 “problem” is this: When rooting on an
external unit (the exceptional use case), both the hostname
and mount point (full pathname) differ from the usual use
case. Hence, whilst the deduplication works, there is a
2nd set of paths/hosts in the repository, despite the
source (the files being backed-up) being the same set as
usual. To be more explicit, in the usual use case, I get
a snapshot of (names invented) with:

Host=my-machine Path=/home/foo/…

But in the exceptional use case, the snapshot is:

Host=other-name Path=/media///home/foo/…

(I am unsure if the is the HDD serial number,
or (more likely) the /home UUID; that is neither here nor
there.)

That does not look very “healthy” for a restore (e.g.)?

Any suggestions on how to deal with this? That is, I
presume/SPECULATE make the exceptional use case look-like
the usual use case; i.e., use the usual use case hostname
and also the usual use case pathnames?

Thanks & cheers!
-blf-

[rawtaz]

@blf

That does not look very “healthy” for a restore (e.g.)?

It’s not a problem, it’s not “unhealthy”. Can you elaborate on what makes you think so?

Also, I’m curious what you mean by “external unit with a root” and “rooting on an external unit”, as I never heard the term “root” or “rooting” in this context before. Do you simply mean the act of storing the repository on those units?

Here’s a couple of pointers that might help:

• You can specify the hostname to use with restic backup using the -H, --host hostname flag.
• When using restic forget you can specify filter which host’s snapshot should be processed using the --host host flag.
• When using restic forget you can specify how snapshots should be grouped using the -g, --group-by string flag.
• There’s also a -H, --host flag to the restic restore command if needed.

So the host is not a problem. The paths aren’t a problem either - you just happen to have different paths in your snapshots, but you can still forget snapshots that you think are duplicates by using the flags above. As you say, deduplication is still effective.

All of these commands’ flags are explained more in restic help <command> and the official documentation.

[blf]

@rawtaz

@blf

That does not look very “healthy” for a restore (e.g.)?

It’s not a problem, it’s not “unhealthy”. Can you elaborate on what makes you think so?

I’m imaging the following situation:

1.-N. Backups of /home/foo (on internal-HDD) to the same repository on an external-HDD.

N+1. Some changes to /home/foo/…

N+2. DISASTER (note no backup contains the step N+1 changes).

N+3. Boot an external-HDD, fsck internal-HDD, mount internal-HDD read-only.

N+4. Backup of /media//home/foo to the repository, to ensure I have the last changes.

Now, sometime later I want to restore (let’s say to a new HDD).
A simple:

restic --repo=… restore latest --target /some/where/else

would result in a confusing mixture of filenames… AH! Sorry, it just
dawned on me: The “/some/where/else” would replace the “/home/foo” and/or
“/media//home/foo” of the snapshot(s), thus avoiding the confusing
mix of filenames. (Obviously I haven’t tested this restore scenario (only
the backup), because I became very bothered by the complete rescan and
different names generated by the step N+3 backup.)

(Note, b.t.w., `restic forget’ does not enter into the above scenario in
any obvious(-to-me) manner.)

@rawtaz

Also, I’m curious what you mean by “external unit with a root” and “rooting on an external unit”, as I never heard the term “root” or “rooting” in this context before. Do you simply mean the act of storing the repository on those units?

No. Apologies. Here, “root” means “root filesystem”,
and “rooting” means “using that root filesystem” (as the root filesystem).
The external-HDDs are fully capable of serving as
the system’s disk, totally ignoring the internal-HDD;
i.e., they are bootable, and contain a root filesystem.

@rawtaz

• You can specify the hostname to use with restic backup using the -H, --host hostname flag.
  -…

So the host is not a problem. The paths aren’t a problem either - you just happen to have different paths in your snapshots, but you can still forget snapshots that you think are duplicates by using the flags above. As you say, deduplication is still effective.

Bingo! I totally apologise, I failed to notice that option.
(Again, however, note there are no obvious duplicate snapshots in my scenario.)

Ok, presuming my understanding of restic restore --target=DIR is correct,
then Yes, I concur, with the addition of the --hostname=HOST option,
things seem like that ought to work as expected. Obviously, I will test
this (time permitting).

cheers!
-blf-

p.s. Apologies if the formatting of this reply is
mucked-up; I am struggling with the interface.

[rawtaz]

@blf

restic --repo=… restore latest --target /some/where/else

would result in a confusing mixture of filenames… AH! Sorry, it just
dawned on me: The “/some/where/else” would replace the “/home/foo” and/or
“/media//home/foo” of the snapshot(s), thus avoiding the confusing
mix of filenames.

If you did that restore for the two latest snapshots (e.g. by giving their individual snapshot IDs in each their own restic restore run), you’d get the following paths where the restored files are placed: /some/where/else/home/foo and /some/where/else/media/home/foo - that is, restic restores the paths that you backed up into the target path you specify (consider it a “targetPath + backupPath” concatenation).

@blf

(Again, however, note there are no obvious duplicate snapshots in my scenario.)

Yeah, in your last example (at least), the first one has the complete initial backup, and the second one has just a few changes in it (assuming you could successfully read those files off the disaster HDD). In that case you can just restore the latest snapshot and it should cover as much as you can get out of your restore in the end (as everything that was readable and that wasn’t deleted will be in the latest snapshot - at least assuming your disaster HDD didn’t fail in such a way that some files looked like they were deleted when in fact you never deleted them, but that’s outside of this discussion/scope).

@blf

p.s. Apologies if the formatting of this reply is
mucked-up; I am struggling with the interface.

No worries But a tip is that when you want to make the “preformatted text” notation inline, you use two ` signs, not one ` and then a '. That will make it look okay. I edited your reply to correct that, you can look in there by hitting the little edit icon if you want.