Backup on Windows: slow and crashed

I initialized a repo on a USB SSD on E::

SET RESTIC_REPOSITORY=E:\Backups\Felix
restic backup C:\Users\Felix\Syncthing\my_data_raw\tracks --use-fs-snapshot

Then I did a backup:

SET RESTIC_REPOSITORY=E:\Backups\Felix
restic backup C:\Users\Felix --use-fs-snapshot

Version:

restic 0.17.3 compiled with go1.23.3 on windows/amd64

The ETA was really long, many hours longer than when using Macrium Reflect for a full disk backup. Plus, when I came back in the morning, my computer was frozen.

I read that it’s necessary to set an exception in Defender anti virus, but my Defender is completely disabled:

Anything else that I can try? Or should I just go back to Macrium? (which has issues on its own)

1 Like

I don’t use Windows myself so take this with a grain of salt, but are you entirely sure that Windows Defender is really completely disabled? What makes you think that? The text in your screenshot only says that “Cloud-delivered protection” is off, that does not necessarily suggest that the entire Defender is off.

I would look into it if I were you, and see if it’s possible to whitelist restic. See this for more information: Virus and Threat Protection in the Windows Security App - Microsoft Support

1 Like

Because I remember spending some time disabling real-time protection when setting up this installation.

But thanks for pointing out:

OK, let me check. Virus & threat protection settings have real-time protection set to off, so that looks good:

To be on the safe side, I added an exception, following the instruction for setting up restic-windows-backup. Note that I’m not using that set of scripts. My setup is way simpler, see my original post.

I’ll try running it again later, then see what happens.

1 Like

Update, as expected no speed improvement. It is still considerably slower than Macrium reflect, perhaps by a factor of three. But let’s see if there is the crash again.

Something else that I realize is that Restic maxes out two of the four cores in my machine.

1 Like

Restic is cloud backup software really. Ability to use local repo is just by-product:) So of course you will find plenty of software faster than restic in such scenario (backup to local storage). Another thing to keep in mind is that restic can be slow during initial backup but subsequent delta snapshots usually are extremely fast. But YMMV. The best is to test.

IMO use the right tool for the job. If your objective is speed and you already found 3 times faster solution you are happy with then you should use it. Macrium Reflect AFAIK is a disk image backup software hence it can easily outperform restic (which is files based) when you store your backup on external disk.

There are many other factors than speed influencing decision for specific backup solution. All depends on individual requirements and restic does not tick all boxes for everybody,

Because for local backend restic uses by default 2 concurrent connections. You can tweak it if needed as mechanical HDD vs NVME SSD RAID over TB4 definitely require different settings to max up speed.

2 Likes

I agree, except that I’m not happy with Macrium Reflect. I use it as an incremental backup tool. But sometimes, too often, seemingly out of the blue, it decides to spend ages on an increment and backs up huge amounts of data even if not much has changed on disk. Also, their licensing policy is cumbersome. Every time I plug in or plug out my Thunderbolt eGPU, I have to transfer my license to the “new” system. I’m tired of it, and Macrium wastes space on my backup medium.

I’ll let this Restic backup finish, then see how increments go. You’re right, best is to test.

Thanks! That’s good to know. As backup medium I’m using a Samsung T7 Shield, an SSD which interfaces by USB 3.2 Gen.2.

1 Like

Then bump it to 4 (as you have 4 cores anyway). Default is too conservative for SSD. Maybe even more. I would start with 8:)

And overall let’s see how long subsequent backups take. The first one is always slow.

As for backup not finishing make sure that your computer does not go to sleep during restic operations. I do not know Windows but there should be some clever way to prevent system sleep when restic is running.

1 Like

One more tunning tip. Increase pack size. For local repo set it to maximum - I think it is 128MB (check in docs).

1 Like

Thanks! I just set RESTIC_PACK_SIZE=128 and am in the process of repeating the backup. Plan is to let it run over night, then see.

1 Like

What is your external disk filesystem? The best approach is to use native NTFS. ExFAT is suboptimal and often leads to weird problems.

1 Like

Windows icon on task bar → Gear icon (Settings) → System → Power and sleep
It does not matter when the Screens turn off. Change the Sleep to never before the first backup. This can be changed to anything after the first backup. My Windows backup takes a couple of hours for the first backup but only a couple of minutes for any more.

1 Like

Thanks, but my system never sleeps by itself. That was not the reason for the crash.

1 Like

That’s how I was greeted this morning: 2100: Detection error on Storage Device (M.2)

It’s a ThinkPad X1 Tablet 3rd Gen with an eGPU connected by Thunderbolt. And I had the USB SSD plugged into the spare Thunderbolt port of the eGPU. After pressing Esc, I ended up in the computer’s BIOS. A warm reboot didn’t help. I had to shut down the machine with the power button and power it up again.

Obviously, this is a hardware issue. I never had that before. Maybe Restic’s heavy processing caused the SSD to overheat. Or it’s simply a coincidence.

1 Like

Yeap. Does not look like anything related to restic per se but highly likely caused by restic stressing hardware a bit more than daily usage.

1 Like

Yes, exFAT. Could try NTFS.

1 Like

I doubt it will make any difference until you fix your hardware issues. Looks like your SSD connection dies when under heavy load. ExFAT could lead to repo corruption etc. but not to results you see.

1 Like

How so? I don’t see why the target file system should make any difference, as long as it supports a directory structure and decently sized files.

1 Like

Hi Windows user here. I have Windows Defender enabled but ignoring rclone and restic and it works fast for local backups too.

In your backup do you exclude temporary files?

Fast is relative. Did you compare speed to other backup programs such as Macrium Reflect?

No. I want them in the backup.

1 Like

Because it is very basic, good maybe for SD card or moving data between systems not having common OS filesystem. Not so good for intensive use involving thousands of files. It is not “journaled” hence lacking any safety measures present in modern filesystem preventing corruption. It is not much better than FAT32 but max file size limit increased.