I am using restic 0.14 in Linux. I have not been getting any backup estimations or progress indicators. So I wanted to find out if there is supposed to be one on 0.14. For some of my initial large backups with my slow upload speeds, it would run for 2 weeks with no progress output. Is this expected? I ended up pulling up the System Monitor app so I could see my upload speed to know it was still running. I read about this bug, but it’s unclear to me if this bug is present in 0.14 or if it was introduced in 0.15?
I tried increasing the verbosity level but all I could get was an output of every file backed up. I just want some indication my backups are progressing and for it to tell me how much it will be backing up.
For instance, I have around 600G in my Backblaze B2 repository. I’m doing a backup right now that should add 24G. The only output I see before the backup is:
scan finished in 12.695s: 97997 files, 612.027 GiB
I’m guessing this is the total size of my local files including the new 24G I just added. Then there is no output for the whole backup after that. Here is my command:
Can you please update to restic 0.15.1 which is the current version? You can download it from here: Releases · restic/restic · GitHub - Or you can run restic self-update.
When I start a backup I see this when it starts scanning files:
repository beb386f4 opened (version 1)
using parent snapshot 3442ab5e
[0:08] 155285 files 14.704 GiB, total 155286 files 14.704 GiB, 0 errors
After a while it adds a percentage:
repository beb386f4 opened (version 1)
using parent snapshot 3442ab5e
[0:38] 90.31% 508010 files 88.193 GiB, total 664416 files 97.652 GiB, 0 errors ETA 0:04
And when it’s all done it finishes with this:
repository beb386f4 opened (version 1)
using parent snapshot 3442ab5e
Files: 0 new, 0 changed, 664416 unmodified
Dirs: 0 new, 0 changed, 113937 unmodified
Added to the repository: 0 B (0 B stored)
processed 664416 files, 97.652 GiB in 0:48
snapshot 2e75c584 saved
Is the percentage what you are missing? Are you seeing it in 0.15.1 after upgrading to that?
On .14, it seems I am completely missing the line that you show that includes the percentage and ETA:
[0:38] 90.31% 508010 files 88.193 GiB, total 664416 files 97.652 GiB, 0 errors ETA 0:04
At the beginning of my backup it prints:
open repository
lock repository
using parent snapshot 324699dc
load index files
start scan on <my directories>
scan finished in 12.695s: 97997 files, 612.027 GiB
Then it displays nothing else while the backup runs. When the backup is done, it displays:
Files: 643 new, 0 changed, 97354 unmodified
Dirs: 43 new, 4776 changed, 2077 unmodified
Data Blobs: 15435 new
Tree Blobs: 4624 new
Added to the repository: 21.066 GiB (20.844 GiB stored)
processed 97997 files, 612.027 GiB in 8:34:22
snapshot 3578482f saved
So I tried it with 0.15.1 and had the same problem, I never saw that progress line. Then I tried to run it manually from the command line instead of from my bash script and sure enough I did see that line quickly flash on the screen but then it was removed after it finished. I didn’t have new data to add to the repository but if I did I think the progress line would work as expected. So it looks like the problem is in my bash script, not restic. The problem seems to be my attempt to redirect stdout & stderr to a logfile while also displaying the output to the console directly. The problem line is:
# Log all stdout+stderr
exec > >(tee -a $LOGFILE) 2>&1
If I remove this, I did quickly see the progress bar when I run the script. So I’ll try to debug this line in my script. I am sure I found this syntax in some example script, but can’t find where now.
So I have looked at my syntax above and there doesn’t seem to be anything wrong with it. This topic is discussed in detail here: BashFAQ/106 - Greg's Wiki.
So I wanted to ask if anyone else has a way to log all your output to a log file yet still see the progress indication when you run your script manually?
It appears when Restic displays the progress indication it keeps overwriting the same line to update it. I’m guessing piping it to tee must be swallowing that somehow. All the other lines that restic prints work fine. I’m guessing that if restic had an option to print each status update on a new line rather than the same like it would work.
By default, Restic only prints a progress bar if stdout is a tty. If the output of restic is redirected, then there’s no progress bar. To get a progress output in this case, you can set the environment variable RESTIC_PROGRESS_FPS, see Manual — restic 0.15.1 documentation for an explanation.