Directory mapping on repos

Getting Help
1

Hello,

Is there a way to back up a directory that its path changes each run (e.g. /staging/run-{timestamp}/data), but have restic record it at the same fixed path in every snapshot (e.g. /fixed/data)?

I tried:

  • Symlink as the backup argument: restic doesn’t follow it.
  • --stdin/–stdin-filename: only a single file/stream, not a directory tree.
  • mounting the real dir onto a fixed mountpoint: gives a fixed path, but remounting a new source over an already mounted point risks disrupting an existing restic process

Is there a way or accepted pattern for this or is changing path simply unavoidable here?

The why: It is a DB dump folder. The real path has to change per run because a previous run can still be slowly uploading (high-latency backend) when the next run starts and the new run must never delete or replace a directory a prior run is still reading from.

restic 0.19.1

Thanks.

2

What about to backup /staging and exclude everything but run-{timestamp}/**

I think something like this will work:

# restic backup /staging --exclude '/staging/*' --exclude '!/staging/run-{timestamp}'

This way you can dynamically change exclusions per run.

3

What about a bind mount?

mount --bind /actual/target /mnt/backup-source
restic -r /your/repo backup /mnt/backup-source

I know it’s not really what you’re asking for, but I propose it as a solution given I don’t think there’s a native restic one.

4

Not yet. Ability to overwrite Drive/Path on backups · Issue #1376 · restic/restic · GitHub.

I can think of a couple of workarounds though, assuming the goal is “back up my database”.

First, the simple one - just backup /staging/. DB dumps should deduplicate very well, and restic should scan each dump only once (assuming the file metadata doesn’t change in between restic runs). You’ll wind up with more than one dump per snapshot, but should see minimal to no additional storage used.

Second, slightly more complex, --stdin-from-command. restic runs the DB dump and ingests the data directly into a snapshot, bypassing writing anything to a staging dir. This may/may not be possible with your setup, depends whether the process making the DB dump is running on the same machine as restic. Relevant docs page: Backing up — restic 0.19.1 documentation

5

Thanks for the answers.

Bind mount was already mentioned in my first post.

My DB backup tool only can write to a directory since its main trick is dumping parallel, so no --stdin for me. I even disabled its built-in compression so restic can efficiently dedup then compress.

Since there is no direct solution for what i’m trying to do, I seperated my DB backup from file backups, tagged and gated such so it always backs up the same directory.

I gave up on my ability to be able to “handle multiple long-running DB backup uploads that does not touch each other” but in reality if a daily backup needs to run longer than a day there is a fundemental problem and it is not restic’s one.