Restore with numeric uid or with username

Getting Help
1

I just tried to restore a restic backup on a different Linux system. Restic seems to restore the ownership of the files/directories to the numeric UID of the original. On this new system that UID maps to a different user. I would have expected restic to restore using the name of the owner, not its UID.

I can’t seem to find a definitive source stating that this is restics intended behaviour (or not). Is this a bug or intended behaviour?

Both the backup and the restore were done using “restic 0.12.0 compiled with go1.15.8 on linux/amd64”.

2

This is how it works (not just restic but in general). Files in your filesystem are stored with the owner recorded using a numeric ID, not a name, and naturally the same goes for how the files are stored in the repository. Just like the users in your system are tracked by numeric ID, the name is just an addon to that. Check your /etc/passwd for example.

So this is entirely expected, and you’ll just have to chown the files after restore.

3

I know that the filesystem uses numeric ID’s to represent user/group names.

I’m a programmer (Python, C++, etc) and I do some devops (Linux, using Ansible). I have no experience contacting open source communities so forgive me if this is not “the way to go about this”. I’m just trying to ascertain if it would be useful to open a “feature suggestion” for this on Github.

I recently migrated to restic from duplicity. Just found out that duplicity does “restore by name” by default and gives the user the option to “restore by ID”. So that is where my expectation comes from. This sparked my curiosity…

Turns out, borgbackup, rsync and even “GNU tar” restore ownership by name, by default. The latter surprised me, but I just verified it (in Ubuntu 18.04).

On the other hand (for example), “dar” restores by numeric ID and is vocal about it (see: Why DAR does save UID/GID instead of plain usernames and usergroups?).

So I guess the “world of backup programs” is a bit divided on this one.

Personally, I would argue that it makes more sense to restore ownership based on name instead of id. And to give the user the option to restore by numeric ID if they really want to. But I would be very happy with just “the option” to restore by name.

So what do you think? Would it be useful for me to open a “feature suggestion” for this on Github?

Apart from this little edge case I bumped into, I’m very happy with restic. Great work!

4

I personally wouldn’t bother. The reason is that when I restore something, I consider it a given that I should make sure that the ownerships and permissions of the files I restore are what they should be after the restore, so by default I check and if needed modify this. Considering it’s just one or two commands to do that, I personally thing it’s more important to spend time on other features and things in restic.

But that’s me. If you think it’s important, you are of course free to open a feature request for it, if there isn’t one already. Noone will hang you for that :slight_smile:

5

I would not be opposed to a flag that alters this behavior during restore, but I would not want restic to remove the current behavior completely. We use restic for server backups, and the uids are extremely important – the names may not even exist during restore.

6

I find it kind of hard to justify polluting the CLI with yet another option just for this simple thing that is one single command away from “fixing” after restore :slight_smile:

7

I’m not saying I’m going to do the work to implement it, only that I wouldn’t oppose a PR adding such a feature.

1 Like
8

In the Szenarios of cloud servers or hosted bare metals, one often restores using a rescue System.
If then the System usernames would be restored according to the System usernames of the rescue System, it would be a Desaster.

So I am happy the way it is.

If you want to have usernames restored, I think you could mount the snapshot and rsync it to where you want. Rsync maps the username by default.

1 Like
9

If every file/directory you’re restoring has the same owner/group it would be easy to “fix” with a single command. But are all backup sets really like that? Also, this would only be an extra option for “restore” which currently (only :wink:) has 10 options.

I agree. I’m not advocating to remove the numeric restore, it has valid use-cases. But I think the “restore by name” has valid use cases too. For one, it would make restic just a little bit more user friendly, especially for linux users who aren’t system administrators (those do exist right? :grinning:).

That’s a helpful idea but (sadly) it doesn’t work. I guess rsync tries to look up the name of the files in the mounted path which are already “mapped to the wrong user” (for lack of better words to describe this).
So I tried something like this:

restic dump latest / | tar x -C /mnt/data

which does seem to restore the “ownership by name”, but it is much slower (about 1,5 times the runtime of a “normal” restore, in my limited testing).

I’ve also considered using restic --json ls with some jq magic, but that sadly doesn’t output the user/group names. In fact, I don’t know of any way to get restic to show me the ownership names. According to the manual on metadata handling those names are stored in the repository, but there currently doesn’t seem to be a way to get them out. I’m overlooking something here right?

I haven’t found much topics on this forum or Github referencing some form of “restore ownership by name” for restic. Apart from a mention here (Mapping Posix Owner Usernames to UIDs). From the replies in this topic I get the sense that there also doesn’t seem to be much interest in having any form of “restore ownership by name” implemented.

So let me ask you this. Anyone opposed to adding a note to restic’s documentation about the “numeric ownership restore”? Maybe with some hints for those who really need ownership restored by name? I would be willing to create a PR for that.

10

I just had a look how tar is handling this, see e.g.

So it seems that GNU tar saves both UID/GID and user/group names. For extracting, it obviously uses the names if they exist and the UID if not.
This explains why the dump command of @Arjan worked.
There also is an option to change this default behavior and use UID/GID instead.

Now restic also saves UID and GID as well as user and group name.
I think that we should have a reasonable default for restoring but also offer an option to restore to change this default.

2 Likes
11

@alexweiss Any idea how that works with Windows - what is stored for user and group when you back up there? I’m pondering what possibilities there are to restore using user and group names on that platform.

12

I don’t have a Windows system at hand, so I can’t test it. But I just had a short look at the code. It seems that on Windows systems the archiver saves a fixed UID=0 and GID=0 and empty user/group name. The restorer also doesn’t handle file ownership at all.

13

Hi, I stumbled upon this topic and don’t understand the discussion. It would be super cool to support restore by username of course. In a real disaster case where you quickly need to restore complex filesystem hierarchies you don’t have time to care for correct user ids or separate rsync steps, etc.

14

To add another backup tool - compatible with restic repositories - I’d like to mention that rustic also supports restoring by names (per default) as well as UID/GID (option --numeric-id).

Both options of course only differ when names on the backup/restore system don’t match - and in this case opinions may vary about what is the most sensible default to use. Anyway, you can use these rustic features to restore until restic also supports it.

15

Hello,
I’m new to restic and not familiar with how everything works yet, but I think I’ve got the basic principles
I recently bought a new SSD to upgrade my laptop and I wanted to kill two birds with one stone and upgrade my system at the same time. When installing the new system (Xubuntu 24.04), I chose a different username (and hostname) from my previous install.

When I try to restore files from my backup however, their ownership seems to be wrong: all restored files belong to root:root instead of my user name and group.
I checked the snapshot using restic ls and they are correctly shown as belonging to 1000:1000

Is this due to the change of username, which doesn’t make sense if restic uses uid/gid, or is it a consequence of running restic in sudo ?
Most importantly, is there an easy solution, other than manually chowning every directory/file ?

16

@LordBaryhobal Can you describe in more detail the steps you performed to restore and how you inspected the files to conclude that they are owned by root:root instead of the user with UID/GID 1000/1000?

Unless I’m terribly mistaken, restic does restore ownership, so I’d expect your files to be owned by 1000:1000, at least that’s what the code suggests:

func (node Node) restoreMetadata(path string, warn func(msg string)) error {
	var firsterr error

	if err := lchown(path, int(node.UID), int(node.GID)); err != nil {
		// Like "cp -a" and "rsync -a" do, we only report lchown permission errors
		// if we run as root.
		if os.Geteuid() > 0 && os.IsPermission(err) {
			debug.Log("not running as root, ignoring lchown permission error for %v: %v",
				path, err)
		} else {
			firsterr = errors.WithStack(err)
		}
	}

You say you did run the restore as sudo, can you verify that you are really UID 0 when you run sudo the way you did?

In summary, we need more specific information and details from you in order to be of any help.

17

Sure
Backup
I used the following command to backup my system before changing the SSD. (I also backed up /etc and /opt, but I don’t think this is relevant to my issue)
sudo restic backup -r my-repo /home/user1

Check
I used the following command to check the snapshot’s contents:
sudo restic ls -l -r my-repo b382a3ff /home/user1

(sample output)

drwxr-x---  1000  1000      0 2024-06-19 14:17:06 /home/user1
-rw-------  1000  1000      0 2020-12-19 18:30:41 /home/user1/.ICEauthority
-rw-------  1000  1000     51 2024-06-19 14:06:28 /home/user1/.Xauthority
-rw-r--r--  1000  1000   1600 2020-12-19 14:17:12 /home/user1/.Xdefaults
drwxrwxr-x  1000  1000      0 2024-04-16 13:16:40 /home/user1/.android
drwxrwxr-x  1000  1000      0 2024-01-29 07:35:00 /home/user1/.audacity-data
-rw-------  1000  1000  53644 2024-06-19 14:17:06 /home/user1/.bash_history
-rw-r--r--  1000  1000    220 2020-12-19 14:17:12 /home/user1/.bash_logout
-rw-r--r--  1000  1000   4113 2023-11-13 10:41:49 /home/user1/.bashrc

Restore
To restore files, I used the following command (I did it dir by dir to only restore what was really needed, although I could have probably used --include)
sudo restic restore -r my-repo b382a3ff:/home/user1/.android -t /home/user2/.android

But then, ls -al /home/user2/ | grep .android gives the following info:

 drwx------  4 root  root   4096 Jun 23 00:40 .android

Mounting the repo, however, does seem to work and shows the correct owner (i.e. user2:user2)

Running id returns: uid=1000(user2) gid=1000(user2) groupes=1000(user2) ...
Running sudo id returns: uid=0(root) gid=0(root) groupes=0(root)

Let me know if you need anything more

18

Just to be sure: are you doing your check with ls while restore is still running?

I just asked because I think I saw that behaviour one time while restoring. Looked to me as if restic first restores the files itself and then at the end applies metadata and permissions / ownerships.

19

No, I ran it after restore completed, at least when the command returned (maybe some things are done in the background ?)

20

When restic is finished, there are no background tasks.
As said, it was just a thought because of me noticing similar behaviour when playing with ls on restored files while restic restore was still working.