Different log file in Windows and Linux

Getting Help
1

Hello,

I’m setting up Restic on Windows 7 and it works really great :rocket:
The only difference to Linux (Centos 8) is the content of the log file.
All commands are run from shell / batch script.

Linux

restic backup --repo /backup /source --cleanup-cache --verbose --verbose >> /restic.log 2>&1

Result

open repository
lock repository
load index files
using parent snapshot 5f9668c4
start scan on [/source]
start backup on [/source]
new       /source/file.txt, saved in 0.044s (100 B added)
unchanged /source/file.jpg
...

Windows

restic backup --repo C:\Backup C:\Users --cleanup-cache --verbose --verbose >> C:\restic.log 2>&1

Result

open repository

e[2Klock repository

e[2Kload index files

e[2Kusing parent snapshot cf8362f4

e[2Kstart scan on [C:\Users]

e[2Kstart backup on [C:\Users]

e[2K[0:00] 0 files, 0 B, 0 errors

e[2Kunchanged /C/Users/MyUser/Config.xml

e[2K[0:00] 0 files, 0 B, 0 errors

e[2Kunchanged /C/Users/MyUser/test.html
...

Why is the progress indicator saved into the log file on Windows?
Each line contains {ESC}[2K at the beginning which is an escape character for “Clear entire line”

Thanks for your help!

2

The output file is the same just that the program which you are using on Windows 7 to look at the log file does not know what to do with UTF-8 characters. I presume that you know Windows 7 is no longer supported by Microsoft without a paid support contract.
If you are willing to install PowerShell, this is what I do on Windows 10:

restic --restic-options | Out-File -FilePath stdout.log -encoding ascii

It pipes the stdout and converts it to ascii.
Another option is to transfer the file from the Windows 7 to your unix system and then just use it the same as you other log files.
Another option is to install python and read the output

with open(readstr, "r", encoding='utf-8') as readfile:

Also be sure to tell your antivirus program to ignore restic.exe otherwise things will be slow. You do not say which version of restic you are using.

Using version 0.11 or later then the option “–use-fs-snapshot” can be used to be able to read open files. This works with Windows 10 but may not work with Windows 7. Be sure to test a recovery before you need it.

3

I’m seeing the same characters when running a backup without --quiet on Windows 10. Viewing the log file with Notepad++ (which knows how to handle UTF-8) on Windows or viewing it on Linux doesn’t make a difference. Converting the log file to ANSI after it has been written doesn’t make a difference either.

4

Thanks for the suggestions, I will try the conversions you suggested.

No, the windows one also has a line with progress text after each line with filename.

5

It’s not a UTF-8 characters, it’s ANSI Escape codes.

Windows can’t handle this natively, and older versions of restic don’t write these codes if you redirect output to a file, but versions after 0.9.6 started to write them.

Personally, I use a modified restic version in which this “feature” is disabled.

6

Thanks, then I remembered correctly that it did not happen in older versions.

I repeat my question: why is progress text saved in log file?
If I remove the special characters and line breaks, I get

open repository
lock repository
load index files
using parent snapshot ...
start scan on ...
start backup on ...
[0:00] 0 files, 0 B, 0 errors
unchanged Config.xml
[0:00] 0 files, 0 B, 0 errors
unchanged test.html

and on Linux it is

open repository
lock repository
load index files
using parent snapshot ...
start scan on ... 
start backup on ...
unchanged Config.xml
unchanged test.html

which is correct and what I want.

Any ideas on that?

7

This comment say that the status (current progress) must be disabled for redirected output, but that’s a lie: on Windows, status is disabled only when redirecting to a pipe.

You can modify this file and recompile restic (as I did) or report issue on github.

8

I attempted to run exactly what you did to get rid of the wild characters in the log output but it looks the same for me. Windows 10, Pwsh 7.1.

I ran:

restic -r $resticRepo -p $resticPass backup C:\users\User\Files\ | Out-File C:\temp\files.log -Encoding ascii

If that doesn’t work I suppose I could work around it with the Start-Transcript Cmdlet.

9

hi,
Your Out-File syntax seems to be incorrect. Change it to:

Out-file -FilePath C:\temp\files.log -Encoding ascii

For details see: https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/out-file?view=powershell-7.1

10

You don’t necessarily have to specify the -FilePath option. Path is required on that Cmdlet so PowerShell knows what to expect in that position.

It’ll be the same whether you include it or not.

11

The issue has been fixed in PR Fix windows terminal output for mintty by MichaelEischer · Pull Request #3325 · restic/restic · GitHub which is not yet included in a released restic version. In the mean time you could build restic yourself or use a beta version from restic beta releases (/) .

2 Likes
12

Ah good to know. Thank you.

I already went ahead and worked around it with PowerShell transcripts so I’ll wait for it in a release before I switch up my script.

Edit: I’m a dummy… That’s not how transcripts work. I’ll just deal with the weird output until it’s resolved in a release.