Windows junction backup with go1.23 or later

According to Go, Backwards Compatibility, and GODEBUG - The Go Programming Language.

Go 1.23 changed the mode bits reported by os.Lstat and os.Stat for reparse points, which can be controlled with the winsymlink setting. As of Go 1.23 (winsymlink=1 ), mount points no longer have os.ModeSymlink set, and reparse points that are not symlinks, Unix sockets, or dedup files now always have os.ModeIrregular set. As a result of these changes, filepath.EvalSymlinks no longer evaluates mount points, which was a source of many inconsistencies and bugs. At previous versions (winsymlink=0 ), mount points are treated as symlinks, and other reparse points with non-default os.ModeType bits (such as os.ModeDir) do not have the ModeIrregular bit set.

Junctions no longer have os.ModeSymlink flag unless //go:debug winsymlink=0 is set. Causing an error such as

error: restictest\junctosys: unsupported file type "irregular"

for a junction like

<JUNCTION>     JUNCTO~1     junctosys [C:\Windows\System32]

which results in junctions not backed up according to this fix.

Currently, go1.22 is used to build a release binary for Windows, so junctions are backed up and restored as symlink, however if go1.22 is dropped of support(go1.24 is released), will backing up junctions in Windows become unavailable later? Or will there be a improvement for the Windows backup and restore implementation to support Windows junction(maybe a more proper implementation to backup and restore as junction instead of symlink in Windows(fallback to symlink for non-Windows platforms))?

Thank you for identifying this. Every year at this time I create a new empty repository and do a backup. This year there were a bunch of errors related to “irregular” files but I thought that had been resolved with restic 0.16 or 0.17. I am on 0.17.3 but needed to exclude
C:\users*\appdata\local\Microsoft\WindowsApps** or something like that. After a check and then a purge things were fine.

Do you use a built version of restic by yourself with go1.23+? If you’re using a released version, I think the issue is from something else.

The precompiled version would be downloaded from github.
PS C:\Users\X\Documents\utilities\restic> .${RESTICEXE} version
restic 0.17.3 compiled with go1.23.3 on windows/amd64

So the junction support is already broken with the released binary…, thank you for indicating me that!

Edit: I found out that the dockerfile is not actually used to build release binary, they build it separately with winsymlink=0 is set, which make junctions still backed up and restored as a symlink.
image

It was the reason why manually built binary cannot handle junctions, while release binary can.

1 Like

The release builds are built using Go 1.23 without setting any special options. What’s happening here is that the Go version in go.mod determines which of those go:debug are enabled or not.

This will change once the minimum required Go version changes to 1.23, which will still take some time.

The link targets were never backed up, so the only change is that the link becomes missing.

We can add support if someone comes up with a proper concept how to store the necessary information (no idea what information is required by Windows) and contributes an implementation. Feel free to open an issue on Github though.

1 Like

Now I understand why does building a binary without modifying go.mod works, whereas building binary after changing dependency does not work.

Do you mean that the contents in the directory pointed by junction are not backed up by this? I think it is normal to only backup a metadata of a junction point(to avoid possible circular reference). I meant that the junction will not be restored as symlink(which should work for most cases, even though is has some differences).

The related information for this actually exists in the golang source code itself, reparse_windows.go. It can be implemented using DeviceIoControl function (ioapiset.h) - Win32 apps | Microsoft Learn with FSCTL_GET_REPARSE_POINT - Win32 apps | Microsoft Learn and FSCTL_SET_REPARSE_POINT - Win32 apps. Unfortunately, we cannot get ReparseTag directly from os.Lstat(actually it already has it) to determine if a path is a reparse point as os.Fileinfo.(*os.fileStat).ReparseTag is not exported. So, It will need to be collected with GetFileInformationByHandleEx as it is done internally.

FYI, There is an existing proposal for this, proposal: os: add ReparsePointTag · Issue #70735 · golang/go · GitHub.

Exactly. I just wanted to make that clear as there have been a few requests to support following symlinks/junctions during backup.

I meant how that information can be properly stored in restic’s data format. I’ve stumbled across the reparse point handling while working on fs: filehandle-based File by MichaelEischer · Pull Request #5160 · restic/restic · GitHub . As long as there’s no way to reuse the go standard library code, we can just include and modify the relevant snippets.