Windows: Repo across multiple external USB-SSDs?

Getting Help
1

Hi,

actually I’m dealing with attempts to repair a 2+ TB repo on external SMR HDD that shows poor performance. I did a few operations ( restic rebuild-index, chkdsk /f, heal missing data by rerunning backups, several check runs). rebuild-index and check operations took many hours. There seems to be at least one further rebuild-index necessary…

Idea: Copy repo temporarily on SSD storage in order to speed things up and be safer to not make things worse than they already are. (generally a good idea to work on copy rather than original repo when trying to repair things…)

Problem: My biggest SSD has only 2 TB of storage capacity. My repo is bigger…

Question: Is there a way to distribute the repo across 2 external SSDs? (by means of restic or Windows or both?)

Any idea welcome.

Thanks!

2

Not using restic per se, but you should be able to copy e.g. half of the directories that store data in the repository to the SSD and then mount those dirs back into/over theor original path in the repository, and that way have at least some of the repository be accessed over the faster drive. Just make sure that when you’re done you copy them back from the SSD to get a complete repository in one single place.

3

What kind of “mount” on Windows should be compatible with restic?

“mklink /j” seems to be incompatible (restic check on a test repo reported missing packs which is not the case with contents [back] on same drive)

4

I take it you use Windows. It would be much simpler with a more suitable OS :slight_smile:

Unless I’m mistaken, there are three options to mklink in Windows, being /D for a symbolic link, /J for a directory junction, and /H for a hard link. If neither of these work I’m probably out of ideas, short of mounting

I don’t suppose you have WSL or can set up a VM for this task, if neither of the above link types work?

5

What about Rclone’s union remote? I used it once when copying files into two flash drives. For my use case, I found the mfs (most free space) policy more useful than the default epmfs (existing path first, then most free space) for the create policy.

1 Like
6

Just checked it:

  • copy all repo files from FLASH-drive to SSD except data subdirs from 80 to FF
  • create junctions by mklink /J for subdirs from 80 to FF
  • run restic check --read-data

no errors were found

1 Like
8

Thanks for checking. Seems to work here now, too. (1st POC test might have been on FAT partition or alike…)

By now instead of continuing to try to repair my big repository, I created new smaller ones.

In case I find time to continue my work on the broken repository, I’ll post feedback here and mark the solving post according to my results.

1 Like
9

Hi @bazinga I’ve also tried to run restic on local rclone union remote, but when trying to init I get the following error:

rclone: 2024/02/01 22:36:33 ERROR : : Create repo failed to Mkdir: object not found
Fatal: create repository at rclone:restic_union: failed: server response unexpected: 500 Internal Server Error (500)

I did run the following restic-command:

D:\restic\restic.exe --repo "rclone:restic_union:" --password-file path/to/file --option rclone.program=../rclone/rclone.exe init

What am I doing wrong?

10

Nobody can guess as you do not share anything really… but only what does not work.

Create new thread and provide all details. Then somebody might be able to help.