I just recently discovered restic while working on a backup automation. It seems to be doing a great job, really a handy tool. I however came to a point where I am stuck and things don’t work as expected.
I have this scenario of 3 servers:
- server #1: responsible for calling restic, might be a docker image for instance, running a python script that remotely connects over SSH to any other server making use of the paramiko python library, invoking a restic backup and then cloning the repository to server #3, all being executed on the server #2 over SSH (linux machine)
- server #2: the source server where all data are stored (Windows machine)
- server #3: the destination server for the cloned repository (any OS)
At first sight it might seem to be overcomplicated but needing to clutter cron jobs all over the network and rather centralizing them to a single machine is very well sustainable.
So server #1 opens an SSH session at server #2 and executes a restic backup command, but for the time being having the repository checked might be sufficient as it behaves the same. The example command be (command #1):
set "RESTIC_PASSWORD=pass" && set "RESTIC_REPOSITORY=repo" && restic check
This command works just fine when run on server #2 for local repository. Trying to run the same command from server #2 seeing the list of snapshots on server #3 like this (command #2):
set "RESTIC_PASSWORD=remote_pass" && set "RESTIC_REPOSITORY=sftp:remote_user@remote_host:remote_path" && restic check
makes restic hang infintely. I can run any command, I can also invoke “restic help” for example and the command exists fine. The issue seems to be somewhere in the “sftp” protocol.
Running the command #2 manually connecting over SSH to server #2 works fine.
- connect to server #2 over SSH
- run
set "RESTIC_PASSWORD=pass_repo_server#3" && set "RESTIC_REPOSITORY=sftp:user_server#3@host_server#3:path_server#3" && restic check
Running it from the script does not althought it is the very same command only run from server #1 over SSH:
- run
ssh user_server#2@host_server#2 "set \"RESTIC_PASSWORD=remote_pass\" && set \"RESTIC_REPOSITORY=sftp:remote_user@server#3:remote_path\" && restic check"
I turned on debugging to see whether I might get some hints from the log file. This is what I can see running the command with a clean log:
2024/11/26 16:08:53 restic/main.go:137 main.main 1 main []string{"C:\\ProgramData\\scoop\\apps\\restic\\current\\restic.exe", "check"}
2024/11/26 16:08:53 restic/main.go:138 main.main 1 restic 0.17.3 compiled with go1.23.2 on windows/amd64
2024/11/26 16:08:53 restic/global.go:583 main.innerOpen 1 parsing location sftp:remote_user@remote_host:restic/repo
2024/11/26 16:08:53 restic/global.go:578 main.parseConfig 1 opening sftp repository at &sftp.Config{User:"remote_user", Host:"remote_host", Port:"", Path:"restic/repo", Layout:"", Command:"", Args:"", Connections:0x5}
2024/11/26 16:08:53 sftp/sftp.go:143 sftp.Open 1 open backend with config sftp.Config{User:"remote_user", Host:"remote_host", Port:"", Path:"restic/repo", Layout:"", Command:"", Args:"", Connections:0x5}
2024/11/26 16:08:53 sftp/sftp.go:61 sftp.startClient 1 start client ssh [remote_host -l remote_user -s sftp]
In order to eliminate the env variables I also tried this command as an alternative:
restic --password-file "pass.txt" --repo "sftp:remote_user@remote_host:remote_path" check
Again it worked fine when run directly on the server #2 or via server #1 over interactive SSH connection. I searched online inclundig this forum to no avail. Also tried
ssh user@host -s sftp
as suggested in some topic in here but that worked okay, just like running sftp interactively.
Does anyone have any idea what could be causing the hanging sftp session?
This is the output of restic version (also seen in the log):
restic 0.17.3 compiled with go1.23.3 on windows/amd64