What's the intended stdout format with a --json backup?

Getting Help
1

If you use the --json flag with backup you get tons of intermediate output as the job progresses. The penultimate line is then a “summary” JSON string/structure. Each output line is followed by an empty line.

This is OK… so (in Python) I thought to strip() the penultimate “summary” line and json.loads(...) it. But that fails. Examining the characters at the start of that line is puzzling:

27 (escape), “2”, “K”, “[”. And only then is the next character the “{” marking the start of the JSON string. This looks like some specialist formatting structure. Can anyone say what it is and how such sequences of JSON lines are meant to be treated?

2

I split by lines, since each line is a single json. And I drop all lines except dict['message_type'] == 'summary'.

Here is an example excerpt, since I also do it with python:

    result_dict = {}
    for line in restic_output.decode('utf-8').splitlines():
        temp_dict = json.loads(line)
        if 'message_type' not in temp_dict:
            # Not valid
            continue
        if temp_dict['message_type'] != 'summary':
            # We only care for summary
            continue
        result_dict = temp_dict

Also you can decrease the progress by adding something like this to your restic process’ env:
'RESTIC_PROGRESS_FPS': '0.01666'

3

My question is quite specific, about where this specfic formatting comes from.

It’s not a general question about how to process strings using Python.

4

Oh, sorry then. I didn’t have those mentioned characters though. I’m also using loads() & it doesn’t fail.

1 Like
5

No problem. I should’ve mentioned the OS: W10. This might have something to do with it.

Ah… just upgraded from 0.12.0. to 0.14.0. This problem appears to have vanished.

1 Like
6

That sounds like Log file on Windows contains progress status text, escape characters and extra line breaks · Issue #3111 · restic/restic · GitHub which was fixed in restic 0.12.1

1 Like