Suddenly getting 'Numerical result out of range' warning when backing up from Lustre source

Getting Help
1

Hi all,

  • Running restic 0.18.1
  • Backing up from a Lustre filesystem

So I have this backup cron routine that runs every night. Everything was running fine for months, but in the last few days I started getting those warnings:

error: incomplete metadata for <redacted_filepath>: xattr.list <redacted_filepath>: numerical result out of range

The strange thing is that couple of days ago, it started outputting this warning for a few files. Then yesterday, the number of files with the warning increased and today more files are giving this warning. And all this files exist in the source for more than a year and backup was all fine.

I saw this thread that the fellow user had the same warnings: Numerical result out of range - but since it was a SMB problem and I am backing up from Lustre, I decided to check if you could give me some light on this.

Anyways, thank you very much!

2

For the start could you try with the latest restic release? You are using some outdated one.

3

Hi!

Updated restic to 0.19.1, but the xattr errors still pop up.

Also tried restoring the files, and all of the restored ones had no xattr at all.

The xattrs on the source files are of the following types:

lustre.lov
system.posix_acl_access
trusted.link
trusted.lma
trusted.lov

The one that actually matters is the system.posix_acl_access, the lustre/trusted xattrs are not needed when we have to restore any backup files.

The error is happening for around 500 files in the backup source. The xattr on all of the other files have no problem.

Do you have any idea of what can this be?

Thanks again!

4

At least we can rule out this one.

Not familiar with Lustre filesystem but quick Google AI tells me that it happens with many other tools (getfattr, setfattr, rsync) and solution would require restic’s buffer-growth logic to dynamically retry with a larger allocated block when ERANGE is encountered.

You will need to find somebody familiar with Lustre and golang to investigate it in more details. Maybe there are other this forum readers able to help.

5

Hi @kapitainsky, thanks a lot for your time and input, but I found out what the problem was.

We started backing up to another repository the same sources as redudancy, but on a s3 backend. The thing is that the cron jobs were running simultaneously and the read-concurrency option was set to 64 on both backup scripts. So it was overloading the restic cache and/or lustre I guess.

I’ve tested disabling one of them and everything went smoothly, without warnings. Also worked when reducing the value of read-concurrency on both scripts.

Thanks again!

6

Thank you for sharing the solution!