Backing up paths and command output at the same time

Getting Help
1

I’m trying to do something like:

restic -r /path/to/repo backup \
    /path/one \
    /path/two \
    --stdin-filename three --stdin-from-command command-1 \
    --stdin-filename four --stdin-from-command command-2

This doesn’t work for a bunch of reasons. My understanding is that multiple paths is fine, but --stdin-from-command can’t be used alongside any other source, including a second --stdin-from-command. I also think I understand that if I split it out into multiple restic invocations, that will create snapshot clutter and extra interactive prompting.

My current approach is:

command-1 > /path/three
command-2 > /path/four
restic -r /path/to/repo backup \
    /path/one \
    /path/two \
    /path/three \
    /path/four

Which has the downside of always appearing to restic to be newly updated, bloating backups. I can improve that with some manual change detection scripting, but, before I get into that, two questions:

Do I understand this correctly, and is there a better approach?

(restic version: restic 0.18.0 compiled with go1.24.1 on darwin/amd64)

2

If the content of the command outputs is the same most of the time, it will be deduplicated by restic, so the additional backup storage per snapshot will be minimal.

1 Like
3

Ah, thank you. I tested for this, but coincidentally used a filesize that almost exactly matched the overhead increase over a noop backup. With a little more testing, I’ve confirmed for myself what you stated. It seems that it can result in a few kB extra backup size, but the overhead doesn’t scale with filesize, and maybe is nonexistent in real backup scenarios.

With this deduplication, I think the temporary file approach is fine.

In case anybody’s curious, I tested a single directory with three random data files, sized 1 MB, 10 MB, and 100 MB, after each change backing up the entire directory, and got the following results in bytes:

  • init: +622
  • add dir, 1mb: +1001552
  • no change: +423
  • touch 1mb: +1355
  • add 10mb: +10002253
  • no change: +426
  • touch 1mb: +1522
  • touch 10mb: +1517
  • touch 1mb, 10mb: +1503
  • add 100mb: +100015441
  • no change: +428
  • touch 1mb: +3910
  • touch 100mb: +3902
  • touch 1mb, 100mb: +3907
4

The plan is to add a way to combine snapshots in the future: Option to create cumulative snapshots · Issue #4804 · restic/restic · GitHub . That will allow combining data from multiple sources in a single snapshot.

2 Likes