So, yesterday I put my big restic backup job in a cron script, obviously redirecting stderr/stdout to a file so I could get at them while the backup is running instead of getting them only in the very large resulting cron email when the job finishes.
But, when I did that, I unexpectedly lost the “progress” messages that restic generates to the terminal, ie the ones like these are no longer being generated:
[11:07:59] 81.18% 14645061 files 18.689 TiB, total 33188999 files 23.021 TiB, 0 errors ETA 2:34:49
[12:07:59] 85.63% 18086766 files 19.712 TiB, total 33188999 files 23.021 TiB, 0 errors ETA 2:02:12
[13:07:59] 86.81% 22700045 files 19.985 TiB, total 33188999 files 23.021 TiB, 0 errors ETA 1:59:41
I guess restic stops writing them when it senses it is no longer connected to a terminal (perhaps because in that case it has no termcap to reposition the cursor after printing them?). But even without termcap, they would be most useful, even vital here for me to monitor the backup while it’s running.
I’ve checked and restic 0.9.5 has --json, which seems to output this info in a different format even when redirected to a file – but as this log file is not going to be parsed by a computer but rather by me, JSON is too verbose and would just make my job harder.
So, apart from hacking restic’s code to reenable those progress messages even when output is not a terminal, is there any way for me to get these progress messages back?
Based on a quick reading of the source, these messages are sent to stdout. If you’re redirecting stdout somewhere else, you wouldn’t see them even if you re-enabled them somehow.
Restic would have to be changed to open /dev/tty directly to support writing any output at all when you have redirected stdout and stderr somewhere else.
Based on a quick reading of the source, these messages are sent to stdout.
Yeah, and that’s exactly where I want them to go. To make things clear: I need those messages to go to the redirected file, not to /dev/tty – as in a cron job, there’s no /dev/tty to speak of.
Sorry for not making my issue clearer in the first place.
Screen creates a TTY, which is what restic sees. Restic does not know or care what screen does with the progress information, it just cares that the output is to a TTY. Restic will send progress information to screen; screen will distribute it other places.
You can run screen -r backup to attach to the screen session to monitor progress in real-time. (To detach the screen session and leave restic running, press CTRL + a followed by d while attached to the screen session. You can re-attach later by running screen -r backup again.)
A log file is also generated based on the -L argument, so you have a record of the output even after restic ends.
The log file is appended to, not overwritten!
Writes to the log file are buffered, so e.g. tail -f will not scroll the log in realtime; attach to the screen session to monitor a live process instead.
Thanks again @cdhowie.
I actually thought about doing that with screen – actually I run screen all the time, it’s the first thing I run after login on every system of mine – but it sounds kinda “kludgy” to say the least But anyway, if that’s the only way, perhaps I will do exactly that.
The same could be said about using the handle of a large screwdriver as a hammer
Seriously, IMHO the best way to get the output of program to a file is by having the program actually write to a file and IMHO the Unix way of doing that without changing the program is using a redirection. If said program doesn’t support it, perhaps involuntarily as I think is the case with restic, then OK I guess we can use pty-faking like the one provided by screen. But the best way (IMHO) would be for restic to treat the case where the output is not a TTY better, ie by writing the progress information without the termcap control codes.
Just my $0.02 – and thanks again for giving me a full recipe for kludg^H^H^H^H^Hdoing it with screen so I don’t have to go look it up
I think you are on MacOSX, so I don’t know whether this Linux solution will work for you. Probably yes, but it looks like script for OSX uses different flags than script for Linux.
For literally decades, people have been discovering edge cases where they want to preserve ‘interactive mode’ behavior while programs are redirected. I remember a lot of talk about this in the 90’s when some guy wrote a program to play nethack for him, and he released a suite of tools (ptyget, ptybandage, etc) that could “fake out” a program to make it think it was talking to a tty.
Those are probably still around and might do what you want, but you could get the functionality you are looking for with the script command, which is part of util-linux. Note that all of the cursor control codes will be included in the logfile. I successfully tested the following command on CentOS linux 7:
Neither do I. But then (despite being over 35 years since I started monkeying around with Unix and derivatives) I did not know (or remembered) standard script created a pty; I will soon test it, probably tonight.’
BTW, on Linux you seen to have hit the nail on the head: as per man script:
-c, --command command
Run the command rather than an interactive shell. This makes it easy for a script to capture the output of a program that behaves differently when its stdout is not a tty.`
Kinda ominously, there’s no such helpful observation on the actual MacOS/X man page which I’ve just pasted here [1]; Also, there’s no -c command switch to MacOS/X, it’s specified directly at the end of the command-line.
I remember a lot of talk about this in the 90’s when some guy wrote a program to play nethack for him, and he released a suite of tools (ptyget, ptybandage, etc) that could “fake out” a program to make it think it was talking to a tty. Those are probably still around and might do what you want,
Thanks! I wasn’t aware of that pty* ‘suite’. Will search for it if MacOS/X script doesn’t do what I want.
Didn’t test it yet :-/ Going through a lot of “stuff” here that’s using up all my time.
After reading the above URL, I have high hopes for ptyget. I plan on testing script and then this as soon as I can get my restore troubles solved… stay tuned!
But anyway, if that’s the only way, perhaps I will do exactly that.
