READ_CONCURRENCY, "old" HDD?

[Cal]

• Documentation say you should also be able to set read concurrency by the flag --read-concurrency for the backup command. But I haven’t been able to succeed and it’s not listed in the help text either,is it still true or are the doc not updated?

• Been trying to set read concurrency by env. variable to both 1 and 4 but it doesn’t seem to make any difference (did a dry-run backup, read from old HDD)

• Do you agree that you should only use RC = 1 when reading from HDD? HDD aren’t good at reading multiple files at the same time (heads jumping and wasting a lot of time)

[MichaelEischer]

@Cal

Documentation say you should also be able to set read concurrency by the flag --read-concurrency for the backup command. But I haven’t been able to succeed and it’s not listed in the help text either,is it still true or are the doc not updated?

That documentation is for the current development version. Not for the latest released restic version (0.14.0 at the moment). aka. that feature will be available in the next restic version.

@Cal

Do you agree that you should only use RC = 1 when reading from HDD? HDD aren’t good at reading multiple files at the same time (heads jumping and wasting a lot of time)

You’ll have to try whether it makes any significant difference. But my advice would be to stick to the default unless it makes a large difference.

[Cal]

OK, thanks…I’m running the release version.

I would guess reading from two threads instead of one will decrease the read-performance by a factor 3 to 10, and put unnecessary stress and wear on the HDDs.
Or perhaps this feature if different from what I read/understand?

What is the default, one read-thread? Or is it automatic, perhaps depending on different storage properties?

At least in current release-version I couldn’t see any difference at all, trying 1 or 4 and using dry-run backup. I wasn’t near the source-server but I’m sure I would be able to hear it on the HDDs if it was using multiple simultaneous read-threads.

[MichaelEischer]

@Cal

I would guess reading from two threads instead of one will decrease the read-performance by a factor 3 to 10, and put unnecessary stress and wear on the HDDs.

The estimate is likely wrong. I’d rather expect about 10-20 percent difference, although depending on the data set performance may improve or decrease. The operating system and the HDD try to reduce head movements such that the effect shouldn’t be too large. But as I’ve said, there’s no use in guessing. You’ll have to measure it.

@Cal

At least in current release-version I couldn’t see any difference at all, trying 1 or 4 and using dry-run backup.

The read concurrency is fixed at 2 for the current release version. It is not configurable.

[Cal]

Now I needed to test this IRL

Copy files from a HDD to a SDD
Clear page-cache before I start every test.

Test1: Copy 2 files in serial (only one copy-process at a time), elapsed seconds: 30
(general read-speed about 75MB/s)

Test2: Copy 2 files in parallel (two simultaneous copy processes), elapsed seconds: 40 seconds
(general read-speed about 50MB/s)

I admit the storage-system handle the task more intelligent than expected.

[Schroedingers-Cat]

Do you have a rough estimate when the next restic version will be released?

[rawtaz]

Very soon, but not within a couple of days, if that helps. But you can just go download the latest build from beta.restic.net and use that, no need to wait for the next release!