Possible to move an existing backup directory?

[yorkday]

Been using restic for a little while now on my Synology DiskStation for cloud backup and have been really enjoying its capabilities.

I’d like to know if it is possible to change existing backups to point to a new path?

Today my configuration is:

• Host = “DiskStation”
• Directory = “/volume3/PHOTOS”

The scenario I envision is:

• The backup hostname might change
• The directory path may change, especially if I have to resize/change/move volumes
• I’d like to keep the history of all previous backups even if I move hostname or path

Currently I can’t see a way to change the Directory value of previous backups. I understand with de-duplication, even if I moved the PHOTOS directory to a new path, it would create a new backup snapshot with a new path (e.g. /volume4/PHOTOS), and would not take up any extra space.

Example of today’s snapshots:

ID        Date                 Host         Tags        Directory
----------------------------------------------------------------------
0b065857  2018-04-26 20:06:30  DiskStation              /volume3/PHOTOS
224cc728  2018-05-31 00:03:06  DiskStation              /volume3/PHOTOS
6a3bb0f1  2018-06-30 00:03:14  DiskStation              /volume3/PHOTOS
37787381  2018-07-22 00:03:35  DiskStation              /volume3/PHOTOS
b4b87a03  2018-07-24 00:04:25  DiskStation              /volume3/PHOTOS
590db482  2018-08-14 00:04:54  DiskStation              /volume3/PHOTOS
191d6f2f  2018-08-15 00:02:47  DiskStation              /volume3/PHOTOS
e1d21a5e  2018-08-16 00:03:01  DiskStation              /volume3/PHOTOS
e4bec418  2018-08-17 23:25:52  DiskStation              /volume3/PHOTOS
96b79ab6  2018-08-18 20:28:02  DiskStation              /volume3/PHOTOS
3861e9c0  2018-08-19 00:02:29  DiskStation              /volume3/PHOTOS
d76bc952  2018-08-20 00:03:16  DiskStation              /volume3/PHOTOS

Can I do this?

ID        Date                 Host         Tags        Directory
----------------------------------------------------------------------
0b065857  2018-04-26 20:06:30  NewHostName              /newpath/PHOTOS
224cc728  2018-05-31 00:03:06  NewHostName              /newpath/PHOTOS
6a3bb0f1  2018-06-30 00:03:14  NewHostName              /newpath/PHOTOS
37787381  2018-07-22 00:03:35  NewHostName              /newpath/PHOTOS
b4b87a03  2018-07-24 00:04:25  NewHostName              /newpath/PHOTOS
590db482  2018-08-14 00:04:54  NewHostName              /newpath/PHOTOS
191d6f2f  2018-08-15 00:02:47  NewHostName              /newpath/PHOTOS
e1d21a5e  2018-08-16 00:03:01  NewHostName              /newpath/PHOTOS
e4bec418  2018-08-17 23:25:52  NewHostName              /newpath/PHOTOS
96b79ab6  2018-08-18 20:28:02  NewHostName              /newpath/PHOTOS
3861e9c0  2018-08-19 00:02:29  NewHostName              /newpath/PHOTOS
d76bc952  2018-08-20 00:03:16  NewHostName              /newpath/PHOTOS

If I can’t do the move, I guess I will have to live with?

ID        Date                 Host         Tags        Directory
----------------------------------------------------------------------
0b065857  2018-04-26 20:06:30  DiskStation              /volume3/PHOTOS
224cc728  2018-05-31 00:03:06  DiskStation              /volume3/PHOTOS
6a3bb0f1  2018-06-30 00:03:14  DiskStation              /volume3/PHOTOS
37787381  2018-07-22 00:03:35  DiskStation              /volume3/PHOTOS
b4b87a03  2018-07-24 00:04:25  DiskStation              /volume3/PHOTOS
590db482  2018-08-14 00:04:54  DiskStation              /volume3/PHOTOS
191d6f2f  2018-08-15 00:02:47  DiskStation              /volume3/PHOTOS
e1d21a5e  2018-08-16 00:03:01  DiskStation              /volume3/PHOTOS
e4bec418  2018-08-17 23:25:52  DiskStation              /volume3/PHOTOS
96b79ab6  2018-08-18 20:28:02  DiskStation              /volume3/PHOTOS
3861e9c0  2018-08-19 00:02:29  DiskStation              /volume3/PHOTOS
d76bc952  2018-08-20 00:03:16  DiskStation              /volume3/PHOTOS
12345678  2018-08-21 00:03:16  NewHostName              /newpath/PHOTOS
12345678  2018-08-22 00:03:16  NewHostName              /newpath/PHOTOS
12345678  2018-08-23 00:03:16  NewHostName              /newpath/PHOTOS

In general when I restore, I’m probably going to use the mount feature, and not restore directly back to the existing path - so is it not worth thinking about the old backup directory location if I have to move it in future? Especially given the de-duplication power of restic?

Thoughts? Thanks in advance!

[rawtaz]

Hello,

First of all; Restic deduplicates on the content it backs up, so even if you start backing up a new host or path, you will not have to upload the data again if the data was already uploaded in previous snapshots (and remember, this is on a chunk level, I think around 1 MB or so).

No, you cannot rewrite a previous snapshot to e.g. adjust the path of it. But I don’t think you need to. Your list of snapshots will be like your third example, and that’s fine.

It seems to me like you already understand how it works.

There’s an issue for tracking rewrite of existing snapshots by the way; https://github.com/restic/restic/issues/14

[764287]

@yorkday

Currently I can’t see a way to change the Directory value of previous backups. I understand with de-duplication, even if I moved the PHOTOS directory to a new path, it would create a new backup snapshot with a new path (e.g. /volume4/PHOTOS), and would not take up any extra space.

In general when I restore, I’m probably going to use the mount feature, and not restore directly back to the existing path - so is it not worth thinking about the old backup directory location if I have to move it in future? Especially given the de-duplication power of restic?

When removing snapshots, restic by default groups by hostname and path. If you want to apply 1 policy to all your snapshots you should add tags and use them with --group-by paths only.

Also keep in mind, that the first snapshot with a new directory name requires more time unless you are using --parent to specify the previous snapshot.

[yorkday]

@rawtaz

No, you cannot rewrite a previous snapshot to e.g. adjust the path of it. But I don’t think you need to. Your list of snapshots will be like your third example, and that’s fine.

It seems to me like you already understand how it works.

Thanks for the explanation. I was originally wondering if there was a way to ‘adopt the original backup set’ and effectively change the historical paths (similar to how Arq does it):

Recovering When Your Computer Is Lost or Stolen

But, with your explanation, I understand that changing the historical backup directory path and hostname when moving the backup source to a new location is not important, they are certainly ‘as-was’ backups.

Also with de-duplication, restic should solve this nicely and it won’t be a space problem.

[yorkday]

@764287

When removing snapshots, restic by default groups by hostname and path. If you want to apply 1 policy to all your snapshots you should add tags and use them with --group-by paths only.

Thanks. I learnt the hard way when I backed up multiple directories and applied a ‘forget’ policy which got applied to all paths (luckily I was happy with the policy), but I now make sure I apply it at a specific path level.

@764287

Also keep in mind, that the first snapshot with a new directory name requires more time unless you are using --parent to specify the previous snapshot.

Good to know, thank you.

[764287]

@yorkday

When removing snapshots, restic by default groups by hostname and path. If you want to apply 1 policy to all your snapshots you should add tags and use them with --group-by paths only.

Thanks. I learnt the hard way when I backed up multiple directories and applied a ‘forget’ policy which got applied to all paths (luckily I was happy with the policy), but I now make sure I apply it at a specific path level.

Eh, sorry. This obviously should have been --group-by tags.

[moritzdietz]

@yorkday

I learnt the hard way when I backed up multiple directories and applied a ‘forget’ policy

Always try to make good use of the --dry-run flag when working with forget policies