I’m using restic 0.17.3 compiled with go1.23.3 on windows/386 and there is enough memory and CPU
λ restic -r s3:URL backup C:\data
repository d44bbd18 opened (version 2, compression level auto)
created new cache in C:\Users\Administrator\AppData\Local\restic
no parent snapshot found, will read all files
[0:00] 0 index files loaded
runtime: out of memory: cannot allocate 201326592-byte block (1777664000 in use)
fatal error: out of memory
It happens after uploading 100GB and so data, evetime it’s different, there is 30% of free memory and suddenly boom! Also GOMAXPROCS didn’t work. It still use all cores.
We increased the RAM by 4 times, but it’s not a RAM problem. The issue occurs when the upload is finished and it’s finalizing, despite having 80% of the memory unused. I wanted to leave the error and call stack but it’s too big to fit in this post
restic works reliably for thousands of people (including myself) dealing with repos sometimes order of magnitudes larger than yours.
Of course it is not impossible that you encountered some new bug which manifests itself in your very specific situation (if it was common we would see more reports like yours - we see none) but more likely there is some problem with your overall configuration. What it is exactly is impossible to tell based on your sparse info provided.
For example what is your exact Windows version? You said “windows/386”. Does it mean you use 32bits Windows version?
restic 0.17.3 compiled with go1.23.3 on windows/386
Windows server 2022 Datacenter Ver 21H2 64-bit operating system, x64-based processor
Intel Core Processor (Broadwell, IBRS) 2.40 GHz (4 processors)
10GB free memory
S3 compatible storage (Minio)
Is there any way to fix the repository? hundreds gigabytes files without any snapshot!?
@point5a For me it also seems that you are using the wrong restic binary (if it is the one compiled for 386 CPUs). That could also explain why you are running out of memory, as a 386 CPU is only able to handle 4 GiB of RAM.
But if you already found a solution that works for you, fine.
In lame terms windows_386 means 32 bits version for Intel and windows_amd64 means 64 bits version for Intel. This is not restic authors creative naming conventions but what all industry is using. Naming is a bit confusing but it is what it is.
@point5a Perhaps next time don’t be so quick to state several times that restic is unreliable, is the one at fault, and that the solution is to switch to kopia. Doing so is highly unfair spreading of FUD that neither restic nor any of its wonderful users in the community deserve. I think we all learnt something here
I unmarked your previous switching-to-kopia post from being the solution and marked @kapitainsky’s summary as the solution. Thanks you guys for figuring this out!
To answer this question - as you had already uploaded a lot of data from your backup set, there is a command in restic which you could technically use to create a snapshot out of that. It’s the restic recover command, see restic help recover for more information. I’m not saying it’s something you should do, just letting you know it’s there.
I think this is the already-uploaded data before restic did crash. It will anyway be used when you restart the backup run (also after changing to the correct binary). So my advise is to just re-run another backup until it finishes. To clean up left-overs, run pruneafter that.
Only if you desperately need some of that already-stored data in order to restore lost files, you can try if recover does help you…
Thank you, everyone, for your valuable help. I was under a lot of pressure and feeling angry, but despite that, you were supportive. I have deleted my posts that were not true.