Fatal error: runtime: mcall called on m->g0 stack (AMD)

Getting Help
1

I am experiencing a problem that I believe to be a golang issue, but I suspect I am going to be over my head trying to troubleshoot with them. Is there a restic developer who might be able to lend a hand?

I am commenting on the following issue: runtime: fatal error: mcall called on m->g0 stack on Windows · Issue #67108 · golang/go · GitHub

To be clear, I do not think this is a restic issue, but since I am not a Go developer, I’m not sure how to be helpful to the golang folks. I have also found some similar looking errors from combing through Syncthing issues.

My problem presents with restic_0.17.0_windows_amd64 on Windows Server 2019 running an AMD processor: AMD EPYC 7543P 32-Core Processor

I have not encountered any issues running 0.17.0 on a server with an Intel processor.

I am using the official release binary from:
https://github.com/restic/restic/releases/download/v0.17.0/restic_0.17.0_windows_amd64.zip

Extract the zip to a folder; I am using C:\restic

From Windows GUI, right-click Start and choose Windows PowerShell (Admin) to launch PowerShell as an Administrator.

Then cd to C:\restic and attempt to run the binary.

I have found that after booting the OS, if I repeatedly try to run the binary, eventually it works (usually on the 5th or 6th attempt).

Here is some console output, showing repeated attempts to execute the binary, and eventually the expected output. I also printed systeminfo so you have detailed build and hotfix information.

[Console output exceeded post length limit; will post as successive replies.]

2

Console, Part 1:

Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

PS C:\Windows\system32> cd C:\restic\
PS C:\restic> .\restic_0.17.0_windows_amd64.exe
fatal error: runtime: mcall called on m->g0 stack

runtime stack:
runtime.throw({0x1722ad1?, 0x10?})
        /usr/local/go/src/runtime/panic.go:1023 +0x65 fp=0xc000145c30 sp=0xc000145c00 pc=0x8bd405
runtime.badmcall(0x89a265?)
        /usr/local/go/src/runtime/proc.go:507 +0x1f fp=0xc000145c50 sp=0xc000145c30 pc=0x8c07bf
runtime.badmcall(0x8be899)
        <autogenerated>:1 +0x25 fp=0xc000145c68 sp=0xc000145c50 pc=0x8f5be5
runtime: g 0: unexpected return pc for runtime.badmcall called from 0xc000145c80
stack: frame={sp:0xc000145c50, fp:0xc000145c68} stack=[0x7d0ec04000,0x7d0edff710)


goroutine 1 gp=0xc000066000 m=nil [runnable, locked to thread]:
golang.org/x/text/encoding/unicode.init()
        <autogenerated>:1 +0x1c6 fp=0xc000145e20 sp=0xc000145e18 pc=0xe92906
runtime.doInit1(0x20a1ed0)
        /usr/local/go/src/runtime/proc.go:7176 +0xed fp=0xc000145f50 sp=0xc000145e20 pc=0x8ce06d
runtime.doInit(...)
        /usr/local/go/src/runtime/proc.go:7143
runtime.main()
        /usr/local/go/src/runtime/proc.go:253 +0x327 fp=0xc000145fe0 sp=0xc000145f50 pc=0x8bfdc7
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000145fe8 sp=0xc000145fe0 pc=0x8f0ea1

goroutine 2 gp=0xc000066700 m=nil [force gc (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000069fa8 sp=0xc000069f88 pc=0x8c012e
runtime.goparkunlock(...)
        /usr/local/go/src/runtime/proc.go:408
runtime.forcegchelper()
        /usr/local/go/src/runtime/proc.go:326 +0xb8 fp=0xc000069fe0 sp=0xc000069fa8 pc=0x8bffb8
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000069fe8 sp=0xc000069fe0 pc=0x8f0ea1
created by runtime.init.6 in goroutine 1
        /usr/local/go/src/runtime/proc.go:314 +0x1a

goroutine 3 gp=0xc000066a80 m=nil [GC sweep wait]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00006bf80 sp=0xc00006bf60 pc=0x8c012e
runtime.goparkunlock(...)
        /usr/local/go/src/runtime/proc.go:408
runtime.bgsweep(0xc000074000)
        /usr/local/go/src/runtime/mgcsweep.go:278 +0x94 fp=0xc00006bfc8 sp=0xc00006bf80 pc=0x8aa2d4
runtime.gcenable.gowrap1()
        /usr/local/go/src/runtime/mgc.go:203 +0x25 fp=0xc00006bfe0 sp=0xc00006bfc8 pc=0x89eb45
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00006bfe8 sp=0xc00006bfe0 pc=0x8f0ea1
created by runtime.gcenable in goroutine 1
        /usr/local/go/src/runtime/mgc.go:203 +0x66

goroutine 18 gp=0xc0001041c0 m=nil [GC scavenge wait]:
runtime.gopark(0xc000074000?, 0x18cdd08?, 0x1?, 0x0?, 0xc0001041c0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00010df78 sp=0xc00010df58 pc=0x8c012e
runtime.goparkunlock(...)
        /usr/local/go/src/runtime/proc.go:408
runtime.(*scavengerState).park(0x21443c0)
        /usr/local/go/src/runtime/mgcscavenge.go:425 +0x49 fp=0xc00010dfa8 sp=0xc00010df78 pc=0x8a7cc9
runtime.bgscavenge(0xc000074000)
        /usr/local/go/src/runtime/mgcscavenge.go:653 +0x3c fp=0xc00010dfc8 sp=0xc00010dfa8 pc=0x8a825c
runtime.gcenable.gowrap2()
        /usr/local/go/src/runtime/mgc.go:204 +0x25 fp=0xc00010dfe0 sp=0xc00010dfc8 pc=0x89eae5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00010dfe8 sp=0xc00010dfe0 pc=0x8f0ea1
created by runtime.gcenable in goroutine 1
        /usr/local/go/src/runtime/mgc.go:204 +0xa5

goroutine 19 gp=0xc000104380 m=nil [finalizer wait]:
runtime.gopark(0xc00006de60?, 0x8a70fc?, 0xc0?, 0xe5?, 0x550011?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00006de20 sp=0xc00006de00 pc=0x8c012e
runtime.runfinq()
        /usr/local/go/src/runtime/mfinal.go:194 +0x107 fp=0xc00006dfe0 sp=0xc00006de20 pc=0x89dbc7
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00006dfe8 sp=0xc00006dfe0 pc=0x8f0ea1
created by runtime.createfing in goroutine 1
        /usr/local/go/src/runtime/mfinal.go:164 +0x3d
PS C:\restic> .\restic_0.17.0_windows_amd64.exe
fatal error: runtime: mcall called on m->g0 stack

runtime stack:
runtime.throw({0x1722ad1?, 0xc0003dfc60?})
        /usr/local/go/src/runtime/panic.go:1023 +0x65 fp=0xc0003dfc30 sp=0xc0003dfc00 pc=0x8bd405
runtime.badmcall(0xa300000002145fc0?)
        /usr/local/go/src/runtime/proc.go:507 +0x1f fp=0xc0003dfc50 sp=0xc0003dfc30 pc=0x8c07bf
runtime.badmcall(0x8be899)
        <autogenerated>:1 +0x25 fp=0xc0003dfc68 sp=0xc0003dfc50 pc=0x8f5be5
runtime: g 0: unexpected return pc for runtime.badmcall called from 0xc0003dfc80
stack: frame={sp:0xc0003dfc50, fp:0xc0003dfc68} stack=[0x5de7604000,0x5de77ffd20)


goroutine 1 gp=0xc000066000 m=nil [runnable, locked to thread]:
github.com/restic/restic/internal/feature.init.0()
        /restic/internal/feature/registry.go:16 +0x53e fp=0xc0003dfe20 sp=0xc0003dfe18 pc=0xc3745e
runtime.doInit1(0x20a1c70)
        /usr/local/go/src/runtime/proc.go:7176 +0xed fp=0xc0003dff50 sp=0xc0003dfe20 pc=0x8ce06d
runtime.doInit(...)
        /usr/local/go/src/runtime/proc.go:7143
runtime.main()
        /usr/local/go/src/runtime/proc.go:253 +0x327 fp=0xc0003dffe0 sp=0xc0003dff50 pc=0x8bfdc7
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0003dffe8 sp=0xc0003dffe0 pc=0x8f0ea1

goroutine 2 gp=0xc000066700 m=nil [force gc (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000069fa8 sp=0xc000069f88 pc=0x8c012e
runtime.goparkunlock(...)
        /usr/local/go/src/runtime/proc.go:408
runtime.forcegchelper()
        /usr/local/go/src/runtime/proc.go:326 +0xb8 fp=0xc000069fe0 sp=0xc000069fa8 pc=0x8bffb8
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000069fe8 sp=0xc000069fe0 pc=0x8f0ea1
created by runtime.init.6 in goroutine 1
        /usr/local/go/src/runtime/proc.go:314 +0x1a

goroutine 3 gp=0xc000066a80 m=nil [GC sweep wait]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00006bf80 sp=0xc00006bf60 pc=0x8c012e
runtime.goparkunlock(...)
        /usr/local/go/src/runtime/proc.go:408
runtime.bgsweep(0xc000074000)
        /usr/local/go/src/runtime/mgcsweep.go:278 +0x94 fp=0xc00006bfc8 sp=0xc00006bf80 pc=0x8aa2d4
runtime.gcenable.gowrap1()
        /usr/local/go/src/runtime/mgc.go:203 +0x25 fp=0xc00006bfe0 sp=0xc00006bfc8 pc=0x89eb45
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00006bfe8 sp=0xc00006bfe0 pc=0x8f0ea1
created by runtime.gcenable in goroutine 1
        /usr/local/go/src/runtime/mgc.go:203 +0x66

goroutine 4 gp=0xc000066c40 m=nil [GC scavenge wait]:
runtime.gopark(0xc000074000?, 0x18cdd08?, 0x1?, 0x0?, 0xc000066c40?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00007bf78 sp=0xc00007bf58 pc=0x8c012e
runtime.goparkunlock(...)
        /usr/local/go/src/runtime/proc.go:408
runtime.(*scavengerState).park(0x21443c0)
        /usr/local/go/src/runtime/mgcscavenge.go:425 +0x49 fp=0xc00007bfa8 sp=0xc00007bf78 pc=0x8a7cc9
runtime.bgscavenge(0xc000074000)
        /usr/local/go/src/runtime/mgcscavenge.go:653 +0x3c fp=0xc00007bfc8 sp=0xc00007bfa8 pc=0x8a825c
runtime.gcenable.gowrap2()
        /usr/local/go/src/runtime/mgc.go:204 +0x25 fp=0xc00007bfe0 sp=0xc00007bfc8 pc=0x89eae5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00007bfe8 sp=0xc00007bfe0 pc=0x8f0ea1
created by runtime.gcenable in goroutine 1
        /usr/local/go/src/runtime/mgc.go:204 +0xa5

goroutine 5 gp=0xc000066fc0 m=nil [finalizer wait]:
runtime.gopark(0xc00006de60?, 0x8a70fc?, 0xc0?, 0xe5?, 0x550011?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00006de20 sp=0xc00006de00 pc=0x8c012e
runtime.runfinq()
        /usr/local/go/src/runtime/mfinal.go:194 +0x107 fp=0xc00006dfe0 sp=0xc00006de20 pc=0x89dbc7
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00006dfe8 sp=0xc00006dfe0 pc=0x8f0ea1
created by runtime.createfing in goroutine 1
        /usr/local/go/src/runtime/mfinal.go:164 +0x3d

goroutine 18 gp=0xc000211500 m=nil [select]:
runtime.gopark(0xc000077f78?, 0x3?, 0x8?, 0x1?, 0xc000077f72?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000077e18 sp=0xc000077df8 pc=0x8c012e
runtime.selectgo(0xc000077f78, 0xc000077f6c, 0xc000220d00?, 0x0, 0x0?, 0x1)
        /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc000077f38 sp=0xc000077e18 pc=0x8d0585
go.opencensus.io/stats/view.(*worker).start(0xc000220d00)
        /home/build/go/pkg/mod/go.opencensus.io@v0.24.0/stats/view/worker.go:292 +0x9f fp=0xc000077fc8 sp=0xc000077f38 pc=0x1294dff
go.opencensus.io/stats/view.init.0.gowrap1()
        /home/build/go/pkg/mod/go.opencensus.io@v0.24.0/stats/view/worker.go:34 +0x25 fp=0xc000077fe0 sp=0xc000077fc8 pc=0x1294165
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000077fe8 sp=0xc000077fe0 pc=0x8f0ea1
created by go.opencensus.io/stats/view.init.0 in goroutine 1
        /home/build/go/pkg/mod/go.opencensus.io@v0.24.0/stats/view/worker.go:34 +0x8d
PS C:\restic> .\restic_0.17.0_windows_amd64.exe
fatal: morestack on g0
Exception 0x80000003 0x0 0x0 0x8f0d01
PC=0x8f0d01

runtime.abort()
        /usr/local/go/src/runtime/asm_amd64.s:1157 +0x1 fp=0xc00063fc20 sp=0xc00063fc18 pc=0x8f0d01
runtime.morestack()
        /usr/local/go/src/runtime/asm_amd64.s:593 +0x3d fp=0xc00063fc28 sp=0xc00063fc20 pc=0x8eef7d

goroutine 1 gp=0xc000066000 m=nil [runnable]:
github.com/spf13/cobra.(*Command).ExecuteC(0x21345e0)
        /home/build/go/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:1054 +0x625 fp=0xc00063fe00 sp=0xc00063fdf8 pc=0xc28b05
github.com/spf13/cobra.(*Command).Execute(...)
        /home/build/go/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:1041
github.com/spf13/cobra.(*Command).ExecuteContext(...)
        /home/build/go/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:1034
main.main()
        /restic/cmd/restic/main.go:125 +0x265 fp=0xc00063ff50 sp=0xc00063fe00 pc=0x13ca3e5
runtime.main()
        /usr/local/go/src/runtime/proc.go:271 +0x28b fp=0xc00063ffe0 sp=0xc00063ff50 pc=0x8bfd2b
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00063ffe8 sp=0xc00063ffe0 pc=0x8f0ea1

goroutine 2 gp=0xc000066700 m=nil [force gc (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000069fa8 sp=0xc000069f88 pc=0x8c012e
runtime.goparkunlock(...)
        /usr/local/go/src/runtime/proc.go:408
runtime.forcegchelper()
        /usr/local/go/src/runtime/proc.go:326 +0xb8 fp=0xc000069fe0 sp=0xc000069fa8 pc=0x8bffb8
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000069fe8 sp=0xc000069fe0 pc=0x8f0ea1
created by runtime.init.6 in goroutine 1
        /usr/local/go/src/runtime/proc.go:314 +0x1a

goroutine 3 gp=0xc000066a80 m=nil [GC sweep wait]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00006bf80 sp=0xc00006bf60 pc=0x8c012e
runtime.goparkunlock(...)
        /usr/local/go/src/runtime/proc.go:408
runtime.bgsweep(0xc000074000)
        /usr/local/go/src/runtime/mgcsweep.go:278 +0x94 fp=0xc00006bfc8 sp=0xc00006bf80 pc=0x8aa2d4
runtime.gcenable.gowrap1()
        /usr/local/go/src/runtime/mgc.go:203 +0x25 fp=0xc00006bfe0 sp=0xc00006bfc8 pc=0x89eb45
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00006bfe8 sp=0xc00006bfe0 pc=0x8f0ea1
created by runtime.gcenable in goroutine 1
        /usr/local/go/src/runtime/mgc.go:203 +0x66

goroutine 4 gp=0xc000066c40 m=nil [GC scavenge wait]:
runtime.gopark(0x10000?, 0x18cdd08?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00007bf78 sp=0xc00007bf58 pc=0x8c012e
runtime.goparkunlock(...)
        /usr/local/go/src/runtime/proc.go:408
runtime.(*scavengerState).park(0x21443c0)
        /usr/local/go/src/runtime/mgcscavenge.go:425 +0x49 fp=0xc00007bfa8 sp=0xc00007bf78 pc=0x8a7cc9
runtime.bgscavenge(0xc000074000)
        /usr/local/go/src/runtime/mgcscavenge.go:658 +0x59 fp=0xc00007bfc8 sp=0xc00007bfa8 pc=0x8a8279
runtime.gcenable.gowrap2()
        /usr/local/go/src/runtime/mgc.go:204 +0x25 fp=0xc00007bfe0 sp=0xc00007bfc8 pc=0x89eae5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00007bfe8 sp=0xc00007bfe0 pc=0x8f0ea1
created by runtime.gcenable in goroutine 1
        /usr/local/go/src/runtime/mgc.go:204 +0xa5

goroutine 5 gp=0xc000066fc0 m=nil [finalizer wait]:
runtime.gopark(0xc00006de60?, 0x8a70fc?, 0xc0?, 0xe5?, 0x550011?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00006de20 sp=0xc00006de00 pc=0x8c012e
runtime.runfinq()
        /usr/local/go/src/runtime/mfinal.go:194 +0x107 fp=0xc00006dfe0 sp=0xc00006de20 pc=0x89dbc7
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00006dfe8 sp=0xc00006dfe0 pc=0x8f0ea1
created by runtime.createfing in goroutine 1
        /usr/local/go/src/runtime/mfinal.go:164 +0x3d

goroutine 6 gp=0xc0001b76c0 m=nil [select]:
runtime.gopark(0xc00007df78?, 0x3?, 0x98?, 0x5?, 0xc00007df72?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00007de18 sp=0xc00007ddf8 pc=0x8c012e
runtime.selectgo(0xc00007df78, 0xc00007df6c, 0xc0000e5700?, 0x0, 0x0?, 0x1)
        /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc00007df38 sp=0xc00007de18 pc=0x8d0585
go.opencensus.io/stats/view.(*worker).start(0xc0000e5700)
        /home/build/go/pkg/mod/go.opencensus.io@v0.24.0/stats/view/worker.go:292 +0x9f fp=0xc00007dfc8 sp=0xc00007df38 pc=0x1294dff
go.opencensus.io/stats/view.init.0.gowrap1()
        /home/build/go/pkg/mod/go.opencensus.io@v0.24.0/stats/view/worker.go:34 +0x25 fp=0xc00007dfe0 sp=0xc00007dfc8 pc=0x1294165
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00007dfe8 sp=0xc00007dfe0 pc=0x8f0ea1
created by go.opencensus.io/stats/view.init.0 in goroutine 1
        /home/build/go/pkg/mod/go.opencensus.io@v0.24.0/stats/view/worker.go:34 +0x8d

goroutine 18 gp=0xc0004041c0 m=nil [chan receive]:
runtime.gopark(0x955c05?, 0x154a9c0?, 0x1?, 0x80?, 0xc000077f60?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000077ed0 sp=0xc000077eb0 pc=0x8c012e
runtime.chanrecv(0xc00010c120, 0xc000077fa0, 0x1)
        /usr/local/go/src/runtime/chan.go:583 +0x3cd fp=0xc000077f48 sp=0xc000077ed0 pc=0x88aa0d
runtime.chanrecv2(0xc00010c120?, 0xc000077fc0?)
        /usr/local/go/src/runtime/chan.go:447 +0x12 fp=0xc000077f70 sp=0xc000077f48 pc=0x88a632
github.com/restic/restic/internal/restic.init.0.func1.1()
        /restic/internal/restic/lock.go:404 +0x99 fp=0xc000077fe0 sp=0xc000077f70 pc=0xcd92f9
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000077fe8 sp=0xc000077fe0 pc=0x8f0ea1
created by github.com/restic/restic/internal/restic.init.0.func1 in goroutine 1
        /restic/internal/restic/lock.go:401 +0x1a

goroutine 34 gp=0xc0001041c0 m=6 mp=0xc000400008 [syscall]:
runtime.notetsleepg(0x21cd9a0, 0xffffffffffffffff)
        /usr/local/go/src/runtime/lock_sema.go:296 +0x31 fp=0xc000117fa0 sp=0xc000117f68 pc=0x890771
os/signal.signal_recv()
        /usr/local/go/src/runtime/sigqueue.go:152 +0x29 fp=0xc000117fc0 sp=0xc000117fa0 pc=0x8ec789
os/signal.loop()
        /usr/local/go/src/os/signal/signal_unix.go:23 +0x13 fp=0xc000117fe0 sp=0xc000117fc0 pc=0x955fd3
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000117fe8 sp=0xc000117fe0 pc=0x8f0ea1
created by os/signal.Notify.func1.1 in goroutine 18
        /usr/local/go/src/os/signal/signal.go:151 +0x1f

goroutine 19 gp=0xc000405dc0 m=nil [GC worker (idle)]:
runtime.gopark(0x9a1dd2f48?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000079f50 sp=0xc000079f30 pc=0x8c012e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc000079fe0 sp=0xc000079f50 pc=0x8a0c85
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000079fe8 sp=0xc000079fe0 pc=0x8f0ea1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 50 gp=0xc000580000 m=nil [GC worker (idle)]:
runtime.gopark(0x9a1d1df94?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000113f50 sp=0xc000113f30 pc=0x8c012e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc000113fe0 sp=0xc000113f50 pc=0x8a0c85
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000113fe8 sp=0xc000113fe0 pc=0x8f0ea1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 7 gp=0xc0001b7a40 m=9 mp=0xc0003a6008 [running]:
        goroutine running on other thread; stack unavailable
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 20 gp=0xc000536000 m=nil [GC worker (idle)]:
runtime.gopark(0x9a1dd2f48?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00039ff50 sp=0xc00039ff30 pc=0x8c012e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc00039ffe0 sp=0xc00039ff50 pc=0x8a0c85
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00039ffe8 sp=0xc00039ffe0 pc=0x8f0ea1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 21 gp=0xc0005361c0 m=nil [chan receive]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0003a1e98 sp=0xc0003a1e78 pc=0x8c012e
runtime.chanrecv(0xc00041f620, 0xc0003a1fa0, 0x1)
        /usr/local/go/src/runtime/chan.go:583 +0x3cd fp=0xc0003a1f10 sp=0xc0003a1e98 pc=0x88aa0d
runtime.chanrecv1(0x0?, 0x0?)
        /usr/local/go/src/runtime/chan.go:442 +0x12 fp=0xc0003a1f38 sp=0xc0003a1f10 pc=0x88a612
main.cleanupHandler(0x0?, 0xc000411950)
        /restic/cmd/restic/cleanup.go:24 +0x2a fp=0xc0003a1fc0 sp=0xc0003a1f38 pc=0x138f8aa
main.createGlobalContext.gowrap1()
        /restic/cmd/restic/cleanup.go:16 +0x25 fp=0xc0003a1fe0 sp=0xc0003a1fc0 pc=0x138f845
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0003a1fe8 sp=0xc0003a1fe0 pc=0x8f0ea1
created by main.createGlobalContext in goroutine 1
        /restic/cmd/restic/cleanup.go:16 +0x98
rax     0x17
rbx     0xc0001003c0
rcx     0xc000100008
rdx     0x0
rdi     0xe653814000
rsi     0xc00063fb18
rbp     0xc00063fc38
rsp     0xc00063fc18
r8      0xc00063f8e8
r9      0xc00063fac0
r10     0x0
r11     0x246
r12     0xe6547ff7e0
r13     0x0
r14     0xc000104000
r15     0x1
rip     0x8f0d01
rflags  0x212
cs      0x33
fs      0x53
gs      0x2b
3

Console, Part 2:

PS C:\restic> .\restic_0.17.0_windows_amd64.exe
fatal error: runtime: mcall called on m->g0 stack

runtime stack:
runtime.throw({0x1722ad1?, 0x10?})
        /usr/local/go/src/runtime/panic.go:1023 +0x65 fp=0xc00035f050 sp=0xc00035f020 pc=0x8bd405
runtime.badmcall(0x18d1c720108?)
        /usr/local/go/src/runtime/proc.go:507 +0x1f fp=0xc00035f070 sp=0xc00035f050 pc=0x8c07bf
runtime.badmcall(0x8be899)
        <autogenerated>:1 +0x25 fp=0xc00035f088 sp=0xc00035f070 pc=0x8f5be5
runtime: g 0: unexpected return pc for runtime.badmcall called from 0xc00035f0a0
stack: frame={sp:0xc00035f070, fp:0xc00035f088} stack=[0x438da04000,0x438dbffca0)


goroutine 23 gp=0xc00046ea80 m=0 mp=0x2145fc0 [GC worker (active)]:
runtime.systemstack_switch()
        /usr/local/go/src/runtime/asm_amd64.s:474 +0x8 fp=0xc0004c3f50 sp=0xc0004c3f40 pc=0x8eee88
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1370 +0x1f7 fp=0xc0004c3fe0 sp=0xc0004c3f50 pc=0x8a0d97
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0004c3fe8 sp=0xc0004c3fe0 pc=0x8f0ea1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 1 gp=0xc000066000 m=nil [runnable]:
text/template.goodFunc({0x18f6b50, 0x14d9100})
        /usr/local/go/src/text/template/funcs.go:112 +0xac fp=0xc00063f238 sp=0xc00063f230 pc=0xbfd1ac
reflect.unpackEface(...)
        /usr/local/go/src/reflect/value.go:160
reflect.ValueOf(...)
        /usr/local/go/src/reflect/value.go:3264
text/template.addValueFuncs(0xc000046540, 0xc00035f390?)
        /usr/local/go/src/text/template/funcs.go:92 +0xf3 fp=0xc00063f360 sp=0xc00063f238 pc=0xbfce73
text/template.createValueFuncs(...)
        /usr/local/go/src/text/template/funcs.go:82
text/template.builtinFuncs.func1()
        /usr/local/go/src/text/template/funcs.go:74 +0x5ae fp=0xc00063f3d0 sp=0xc00063f360 pc=0xc036ce
sync.(*Once).doSlow(0x75?, 0xc000090000?)
        /usr/local/go/src/sync/once.go:74 +0xc2 fp=0xc00063f430 sp=0xc00063f3d0 pc=0x8feca2
sync.(*Once).Do(...)
        /usr/local/go/src/sync/once.go:65
text/template.builtinFuncs()
        /usr/local/go/src/text/template/funcs.go:73 +0x2c fp=0xc00063f450 sp=0xc00063f430 pc=0xbfcd4c
text/template.findFunction({0x174c033, 0x2}, 0xc00035f520?)
        /usr/local/go/src/text/template/funcs.go:149 +0x145 fp=0xc00063f4b8 sp=0xc00063f450 pc=0xbfd4c5
text/template.(*state).evalFunction(0xc00063fc00, {0x16eda80?, 0x21345e0?, 0x2?}, 0x47?, {0x18e87f0, 0xc0000461b0}, {0xc00007e040, 0x3, 0x4}, ...)
        /usr/local/go/src/text/template/exec.go:619 +0xa6 fp=0xc00063f5b0 sp=0xc00063f4b8 pc=0xbf8ec6
text/template.(*state).evalCommand(0xc00035fc00, {0x16eda80?, 0x21345e0?, 0x16ef3b6?}, 0xc0000461b0, {0x1512160?, 0x21cc760?, 0x8?})
        /usr/local/go/src/text/template/exec.go:510 +0x1ce fp=0xc00063f6a8 sp=0xc00063f5b0 pc=0xbf7fce
text/template.(*state).evalPipeline(0xc00063fc00, {0x16eda80?, 0x21345e0?, 0xc000058008?}, 0xc000080240)
        /usr/local/go/src/text/template/exec.go:479 +0x125 fp=0xc00063f7a8 sp=0xc00063f6a8 pc=0xbf7945
text/template.(*state).evalCommand(0xc00035fc00, {0x16eda80?, 0x21345e0?, 0xd0?}, 0xc000046180, {0x1512160?, 0x21cc760?, 0xc0000480d0?})
        /usr/local/go/src/text/template/exec.go:514 +0x269 fp=0xc00063f8a0 sp=0xc00063f7a8 pc=0xbf8069
text/template.(*state).evalPipeline(0xc00063fc00, {0x16eda80?, 0x21345e0?, 0x15e9580?}, 0xc0000801e0)
        /usr/local/go/src/text/template/exec.go:479 +0x125 fp=0xc00063f9a0 sp=0xc00063f8a0 pc=0xbf7945
text/template.(*state).walkIfOrWith(0xc00063fc00, 0x13, {0x16eda80?, 0x21345e0?, 0x89a53d?}, 0x18d1c72ac48?, 0xc000046270, 0x0)
        /usr/local/go/src/text/template/exec.go:301 +0xc6 fp=0xc00063fa60 sp=0xc00063f9a0 pc=0xbf63a6
text/template.(*state).walk(0xc00035fc00, {0x16eda80?, 0x21345e0?, 0x18d1c72ac48?}, {0x18e8ac0?, 0xc00007e080?})
        /usr/local/go/src/text/template/exec.go:291 +0x1bb fp=0xc00063faf8 sp=0xc00063fa60 pc=0xbf60db
text/template.(*state).walk(0xc00063fc00, {0x16eda80?, 0x21345e0?, 0xc000092120?}, {0x18e8880?, 0xc000046150?})
        /usr/local/go/src/text/template/exec.go:280 +0x2e5 fp=0xc00063fb90 sp=0xc00063faf8 pc=0xbf6205
text/template.(*Template).execute(0xc000437100, {0x18d6860?, 0xc00010c058?}, {0x16eda80?, 0x21345e0?})
        /usr/local/go/src/text/template/exec.go:224 +0x278 fp=0xc00063fc80 sp=0xc00063fb90 pc=0xbf5e38
text/template.(*Template).Execute(...)
        /usr/local/go/src/text/template/exec.go:207
github.com/spf13/cobra.tmpl({0x18d6860, 0xc00010c058}, {0x174c02b, 0x7c}, {0x16eda80, 0x21345e0})
        /home/build/go/pkg/mod/github.com/spf13/cobra@v1.8.1/cobra.go:184 +0x165 fp=0xc00063fcc8 sp=0xc00063fc80 pc=0xc241a5
github.com/spf13/cobra.(*Command).HelpFunc.func1(0x21345e0, {0x2103460?, 0x18d6a60?, 0x2103460?})
        /home/build/go/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:465 +0x69 fp=0xc00063fd28 sp=0xc00063fcc8 pc=0xc355e9
github.com/spf13/cobra.(*Command).ExecuteC(0x21345e0)
        /home/build/go/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:1122 +0x5d7 fp=0xc00063fe00 sp=0xc00063fd28 pc=0xc28ab7
github.com/spf13/cobra.(*Command).Execute(...)
        /home/build/go/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:1041
github.com/spf13/cobra.(*Command).ExecuteContext(...)
        /home/build/go/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:1034
main.main()
        /restic/cmd/restic/main.go:125 +0x265 fp=0xc00063ff50 sp=0xc00063fe00 pc=0x13ca3e5
runtime.main()
        /usr/local/go/src/runtime/proc.go:271 +0x28b fp=0xc00063ffe0 sp=0xc00063ff50 pc=0x8bfd2b
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00063ffe8 sp=0xc00063ffe0 pc=0x8f0ea1

goroutine 2 gp=0xc000066700 m=nil [force gc (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000069fa8 sp=0xc000069f88 pc=0x8c012e
runtime.goparkunlock(...)
        /usr/local/go/src/runtime/proc.go:408
runtime.forcegchelper()
        /usr/local/go/src/runtime/proc.go:326 +0xb8 fp=0xc000069fe0 sp=0xc000069fa8 pc=0x8bffb8
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000069fe8 sp=0xc000069fe0 pc=0x8f0ea1
created by runtime.init.6 in goroutine 1
        /usr/local/go/src/runtime/proc.go:314 +0x1a

goroutine 3 gp=0xc000066a80 m=nil [GC sweep wait]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00006bf80 sp=0xc00006bf60 pc=0x8c012e
runtime.goparkunlock(...)
        /usr/local/go/src/runtime/proc.go:408
runtime.bgsweep(0xc000074000)
        /usr/local/go/src/runtime/mgcsweep.go:278 +0x94 fp=0xc00006bfc8 sp=0xc00006bf80 pc=0x8aa2d4
runtime.gcenable.gowrap1()
        /usr/local/go/src/runtime/mgc.go:203 +0x25 fp=0xc00006bfe0 sp=0xc00006bfc8 pc=0x89eb45
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00006bfe8 sp=0xc00006bfe0 pc=0x8f0ea1
created by runtime.gcenable in goroutine 1
        /usr/local/go/src/runtime/mgc.go:203 +0x66

goroutine 4 gp=0xc000066c40 m=nil [GC scavenge wait]:
runtime.gopark(0xc000074000?, 0x18cdd08?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00007bf78 sp=0xc00007bf58 pc=0x8c012e
runtime.goparkunlock(...)
        /usr/local/go/src/runtime/proc.go:408
runtime.(*scavengerState).park(0x21443c0)
        /usr/local/go/src/runtime/mgcscavenge.go:425 +0x49 fp=0xc00007bfa8 sp=0xc00007bf78 pc=0x8a7cc9
runtime.bgscavenge(0xc000074000)
        /usr/local/go/src/runtime/mgcscavenge.go:658 +0x59 fp=0xc00007bfc8 sp=0xc00007bfa8 pc=0x8a8279
runtime.gcenable.gowrap2()
        /usr/local/go/src/runtime/mgc.go:204 +0x25 fp=0xc00007bfe0 sp=0xc00007bfc8 pc=0x89eae5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00007bfe8 sp=0xc00007bfe0 pc=0x8f0ea1
created by runtime.gcenable in goroutine 1
        /usr/local/go/src/runtime/mgc.go:204 +0xa5

goroutine 18 gp=0xc0001041c0 m=nil [finalizer wait]:
runtime.gopark(0xc00006de60?, 0x8a70fc?, 0xc0?, 0xe5?, 0x550011?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00006de20 sp=0xc00006de00 pc=0x8c012e
runtime.runfinq()
        /usr/local/go/src/runtime/mfinal.go:194 +0x107 fp=0xc00006dfe0 sp=0xc00006de20 pc=0x89dbc7
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00006dfe8 sp=0xc00006dfe0 pc=0x8f0ea1
created by runtime.createfing in goroutine 1
        /usr/local/go/src/runtime/mfinal.go:164 +0x3d

goroutine 19 gp=0xc0002168c0 m=nil [select]:
runtime.gopark(0xc000077f78?, 0x3?, 0x8?, 0xca?, 0xc000077f72?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000077e18 sp=0xc000077df8 pc=0x8c012e
runtime.selectgo(0xc000077f78, 0xc000077f6c, 0xc000193700?, 0x0, 0x0?, 0x1)
        /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc000077f38 sp=0xc000077e18 pc=0x8d0585
go.opencensus.io/stats/view.(*worker).start(0xc000193700)
        /home/build/go/pkg/mod/go.opencensus.io@v0.24.0/stats/view/worker.go:292 +0x9f fp=0xc000077fc8 sp=0xc000077f38 pc=0x1294dff
go.opencensus.io/stats/view.init.0.gowrap1()
        /home/build/go/pkg/mod/go.opencensus.io@v0.24.0/stats/view/worker.go:34 +0x25 fp=0xc000077fe0 sp=0xc000077fc8 pc=0x1294165
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000077fe8 sp=0xc000077fe0 pc=0x8f0ea1
created by go.opencensus.io/stats/view.init.0 in goroutine 1
        /home/build/go/pkg/mod/go.opencensus.io@v0.24.0/stats/view/worker.go:34 +0x8d

goroutine 20 gp=0xc000216a80 m=nil [chan receive]:
runtime.gopark(0x955be5?, 0x154a9c0?, 0x1?, 0x0?, 0xc000079f60?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000079ed0 sp=0xc000079eb0 pc=0x8c012e
runtime.chanrecv(0xc000080060, 0xc000079fa0, 0x1)
        /usr/local/go/src/runtime/chan.go:583 +0x3cd fp=0xc000079f48 sp=0xc000079ed0 pc=0x88aa0d
runtime.chanrecv2(0xc000080060?, 0xc000079fc0?)
        /usr/local/go/src/runtime/chan.go:447 +0x12 fp=0xc000079f70 sp=0xc000079f48 pc=0x88a632
github.com/restic/restic/internal/restic.init.0.func1.1()
        /restic/internal/restic/lock.go:404 +0x99 fp=0xc000079fe0 sp=0xc000079f70 pc=0xcd92f9
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000079fe8 sp=0xc000079fe0 pc=0x8f0ea1
created by github.com/restic/restic/internal/restic.init.0.func1 in goroutine 1
        /restic/internal/restic/lock.go:401 +0x1a

goroutine 21 gp=0xc00046e700 m=nil [chan receive]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0004c7e98 sp=0xc0004c7e78 pc=0x8c012e
runtime.chanrecv(0xc00041d0e0, 0xc0004c7fa0, 0x1)
        /usr/local/go/src/runtime/chan.go:583 +0x3cd fp=0xc0004c7f10 sp=0xc0004c7e98 pc=0x88aa0d
runtime.chanrecv1(0x0?, 0x0?)
        /usr/local/go/src/runtime/chan.go:442 +0x12 fp=0xc0004c7f38 sp=0xc0004c7f10 pc=0x88a612
main.cleanupHandler(0x0?, 0xc000444ae0)
        /restic/cmd/restic/cleanup.go:24 +0x2a fp=0xc0004c7fc0 sp=0xc0004c7f38 pc=0x138f8aa
main.createGlobalContext.gowrap1()
        /restic/cmd/restic/cleanup.go:16 +0x25 fp=0xc0004c7fe0 sp=0xc0004c7fc0 pc=0x138f845
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0004c7fe8 sp=0xc0004c7fe0 pc=0x8f0ea1
created by main.createGlobalContext in goroutine 1
        /restic/cmd/restic/cleanup.go:16 +0x98

goroutine 22 gp=0xc00046e8c0 m=6 mp=0xc000084008 [syscall]:
runtime.notetsleepg(0x21cd9a0, 0xffffffffffffffff)
        /usr/local/go/src/runtime/lock_sema.go:296 +0x31 fp=0xc0004c9fa0 sp=0xc0004c9f68 pc=0x890771
os/signal.signal_recv()
        /usr/local/go/src/runtime/sigqueue.go:152 +0x29 fp=0xc0004c9fc0 sp=0xc0004c9fa0 pc=0x8ec789
os/signal.loop()
        /usr/local/go/src/os/signal/signal_unix.go:23 +0x13 fp=0xc0004c9fe0 sp=0xc0004c9fc0 pc=0x955fd3
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0004c9fe8 sp=0xc0004c9fe0 pc=0x8f0ea1
created by os/signal.Notify.func1.1 in goroutine 1
        /usr/local/go/src/os/signal/signal.go:151 +0x1f

goroutine 5 gp=0xc000067180 m=nil [GC worker (idle)]:
runtime.gopark(0x21ce5c0?, 0x3?, 0x4?, 0xe2?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00007df50 sp=0xc00007df30 pc=0x8c012e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc00007dfe0 sp=0xc00007df50 pc=0x8a0c85
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00007dfe8 sp=0xc00007dfe0 pc=0x8f0ea1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 34 gp=0xc000500000 m=nil [GC worker (idle)]:
runtime.gopark(0x9f5f67c74?, 0x3?, 0xe0?, 0x88?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000507f50 sp=0xc000507f30 pc=0x8c012e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc000507fe0 sp=0xc000507f50 pc=0x8a0c85
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000507fe8 sp=0xc000507fe0 pc=0x8f0ea1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 50 gp=0xc000582000 m=nil [GC worker (idle)]:
runtime.gopark(0x9f5f67c74?, 0xc000580020?, 0x1a?, 0xa?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000503f50 sp=0xc000503f30 pc=0x8c012e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc000503fe0 sp=0xc000503f50 pc=0x8a0c85
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000503fe8 sp=0xc000503fe0 pc=0x8f0ea1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c
PS C:\restic> .\restic_0.17.0_windows_amd64.exe

restic is a backup program which allows saving multiple revisions of files and
directories in an encrypted repository stored on different backends.

The full documentation can be found at https://restic.readthedocs.io/ .

Usage:
  restic [command]

Available Commands:
  backup        Create a new backup of files and/or directories
  cache         Operate on local cache directories
  cat           Print internal objects to stdout
  check         Check the repository for errors
  copy          Copy snapshots from one repository to another
  diff          Show differences between two snapshots
  dump          Print a backed-up file to stdout
  find          Find a file, a directory or restic IDs
  forget        Remove snapshots from the repository
  generate      Generate manual pages and auto-completion files (bash, fish, zsh, powershell)
  help          Help about any command
  init          Initialize a new repository
  key           Manage keys (passwords)
  list          List objects in the repository
  ls            List files in a snapshot
  migrate       Apply migrations
  prune         Remove unneeded data from the repository
  recover       Recover data from the repository not referenced by snapshots
  repair        Repair the repository
  restore       Extract the data from a snapshot
  rewrite       Rewrite snapshots to exclude unwanted files
  self-update   Update the restic binary
  snapshots     List all snapshots
  stats         Scan the repository and show basic statistics
  tag           Modify tags on snapshots
  unlock        Remove locks other processes created
  version       Print version information

Flags:
      --cacert file                file to load root certificates from (default: use system certificates or $RESTIC_CACERT)
      --cache-dir directory        set the cache directory. (default: use system default cache directory)
      --cleanup-cache              auto remove old cache directories
      --compression mode           compression mode (only available for repository format version 2), one of (auto|off|max) (default: $RESTIC_COMPRESSION) (default auto)
  -h, --help                       help for restic
      --http-user-agent string     set a http user agent for outgoing http requests
      --insecure-no-password       use an empty password for the repository, must be passed to every restic command (insecure)
      --insecure-tls               skip TLS certificate verification when connecting to the repository (insecure)
      --json                       set output mode to JSON for commands that support it
      --key-hint key               key ID of key to try decrypting first (default: $RESTIC_KEY_HINT)
      --limit-download rate        limits downloads to a maximum rate in KiB/s. (default: unlimited)
      --limit-upload rate          limits uploads to a maximum rate in KiB/s. (default: unlimited)
      --no-cache                   do not use a local cache
      --no-extra-verify            skip additional verification of data before upload (see documentation)
      --no-lock                    do not lock the repository, this allows some operations on read-only repositories
  -o, --option key=value           set extended option (key=value, can be specified multiple times)
      --pack-size size             set target pack size in MiB, created pack files may be larger (default: $RESTIC_PACK_SIZE)
      --password-command command   shell command to obtain the repository password from (default: $RESTIC_PASSWORD_COMMAND)
  -p, --password-file file         file to read the repository password from (default: $RESTIC_PASSWORD_FILE)
  -q, --quiet                      do not output comprehensive progress report
  -r, --repo repository            repository to backup to or restore from (default: $RESTIC_REPOSITORY)
      --repository-file file       file to read the repository location from (default: $RESTIC_REPOSITORY_FILE)
      --retry-lock duration        retry to lock the repository if it is already locked, takes a value like 5m or 2h (default: no retries)
      --tls-client-cert file       path to a file containing PEM encoded TLS client certificate and private key (default: $RESTIC_TLS_CLIENT_CERT)
  -v, --verbose                    be verbose (specify multiple times or a level using --verbose=n, max level/times is 2)

Use "restic [command] --help" for more information about a command.
PS C:\restic> systeminfo /fo csv | ConvertFrom-Csv | select OS*, System*, Hotfix* | Format-List


OS Name             : Microsoft Windows Server 2019 Standard
OS Version          : 10.0.17763 N/A Build 17763
OS Manufacturer     : Microsoft Corporation
OS Configuration    : Member Server
OS Build Type       : Multiprocessor Free
System Boot Time    : 8/21/2024, 12:16:08 AM
System Manufacturer : FreeBSD
System Model        : BHYVE
System Type         : x64-based PC
System Directory    : C:\Windows\system32
System Locale       : en-us;English (United States)
Hotfix(s)           : 5 Hotfix(s) Installed.,[01]: KB5041913,[02]: KB4577667,[03]: KB4589208,[04]: KB5041578,[05]:
                      KB5041577



PS C:\restic> .\restic_0.17.0_windows_amd64.exe version
restic 0.17.0 compiled with go1.22.5 on windows/amd64
4

It’s hard to give advice what might be helpful as we don’t really know what to look for (that’s why there has been the question how to reproduce the error, such that the go developer can trigger the crash on their machines and then debug it in detail).

In the golang issue the go race detector was mentioned. You could build restic using go build -race ./cmd/restic and check whether it prints some additional information before crashing.

5

@MichaelEischer I built restic with several different versions of go using the restic/builder (by modifying the GO_VERSION arguments). Can you explain how to build with the race detector, using the Docker container?

I am using:

docker run --rm \
    --volume "$PWD/restic-0.17.0:/restic" \
    --volume "$PWD/output:/output" \
    restic/builder:${GO_VERSION} \
    go run helpers/build-release-binaries/main.go --version 0.17.0 --platform windows/amd64

I’m just not sure where to drop your command, since I am building in Docker. It is sort of an onion that I need to unravel, but it would be easier for you to point me in the right direction.

Thanks!

6

I contributed more information to the golang issue on GitHub, but I am offering the following here for those interested:

  • I compiled restic with every point version of go between 1.20.3 and 1.23.0.
  • I wrote a PowerShell script to execute restic with the version option 20 times.
  • I found that restic compiled with versions of go 1.21.4 and earlier did not panic.
  • The version of restic compiled with go version 1.21.5 panicked during the first 3 of 20 iterations.
  • The problem was observed with builds using golang 1.21.5 through the latest release, 1.23.0.
  • The problem did not present itself on my AMD laptop, CPU: AMD Ryzen 5 PRO 5650U with Radeon Graphics
  • The problem does reliably present itself on AMD server, CPU: AMD EPYC 7543P 32-Core Processor
  • I have never observed an instance where restic built with go 1.21.5+ does not panic, immediately following a reboot, on the affected system.

@MichaelEischer is there any risk running restic 0.17.0 compiled with go 1.21.4 (as opposed to 1.22.5)? If there is a way to test the build, can you please refer me to instructions to run tests?

7

Building a Windows binary using the race detector doesn’t work from within Linux docker containers (at least as far as I can tell). I don’t have a Windows installation at hand right now, so I can’t test the necessary steps.

Something like the following might work. Although there are probably a few syntax errors in it.

docker run --rm --volume "$PWD/restic-0.17.0:c:\restic" --volume "$PWD/output:c:\output" --env GOOS=windows --env GOARCH=amd64 --env CGO_ENABLED=1 --workdir c:\restic golang:1.22.6 go build -race -o c:\output\restic.exe ./cmd/restic
8

I’m building in a Linux docker container, so I’ll probably just abandon that effort for now.

How about the risk of using the binary compiled with go 1.21.4–any reason to believe it is unsafe?

9

Go 1.21.4 misses several security fixes included in later go versions. Our test suite is run against several go versions, including the latest Go 1.21.x version. So, that binary will likely work, however, as it misses several security fixes should only be used in a trusted environment.