I have written some units for scheduled backup using systemd timers and instantiated units. I have 2 instantiated units: one to backup system files, and one to backup my vmdisks. The system backup works fine, my problem is with the vmdisks backup; it seems to run fine, but restic exits with status 203 which puts the unit into a failed state in systemd.
This is the command that systemd runs:
/usr/local/bin/restic backup \
--tag vmdisks \
--verbose --verbose --verbose --verbose \
--exclude-file /etc/restic/vmdisks.exclude \
--files-from /etc/restic/vmdisks.include \
--exclude-caches
I added the extra --verbose flags to debug. Output below.
systemd creates the relevant environment for RESTIC_REPOSITORY and RESTIC_PASSWORD.
Running restic snapshots shows all the snapshots I expect, and I can ls the snapshots and see the files I expect.
Logfile from journalctl. I’ve redacted the actual filenames for confidentiality and readability:
Jul 24 09:47:05 kvm1807 systemd[1]: Starting restic "vmdisks" backup...
Jul 24 09:47:05 kvm1807 restic[11095]: open repository
Jul 24 09:47:05 kvm1807 restic[11095]: lock repository
Jul 24 09:47:06 kvm1807 restic[11095]: load index files
Jul 24 09:47:06 kvm1807 restic[11095]: using parent snapshot 977c071a
Jul 24 09:47:06 kvm1807 restic[11095]: start scan on [/vm-disks/]
Jul 24 09:47:06 kvm1807 restic[11095]: start backup on [/vm-disks/]
Jul 24 09:47:06 kvm1807 restic[11095]: scan finished in 1.227s: 11 files, 267.013 GiB
Jul 24 09:47:06 kvm1807 restic[11095]: unchanged /vm-disks/hdd/XXXA.qcow2
Jul 24 09:47:50 kvm1807 restic[11095]: modified /vm-disks/hdd/XXXB.qcow2, saved in 43.270s (101.061 MiB added)
Jul 24 09:49:27 kvm1807 restic[11095]: modified /vm-disks/ssd/XXXC.qcow2, saved in 139.668s (335.425 MiB added)
Jul 24 09:50:26 kvm1807 restic[11095]: modified /vm-disks/ssd/XXXD.qcow2, saved in 156.803s (281.013 MiB added)
Jul 24 09:50:49 kvm1807 restic[11095]: modified /vm-disks/ssd/XXXE.qcow2, saved in 81.874s (160.754 MiB added)
Jul 24 09:51:06 kvm1807 restic[11095]: uploading intermediate index
Jul 24 09:51:06 kvm1807 restic[11095]: uploaded intermediate index 9381285c
Jul 24 09:51:45 kvm1807 restic[11095]: modified /vm-disks/ssd/XXXF.qcow2, saved in 78.091s (216.208 MiB added)
Jul 24 09:52:35 kvm1807 restic[11095]: modified /vm-disks/ssd/XXXG.qcow2, saved in 106.296s (64.526 MiB added)
Jul 24 09:52:41 kvm1807 restic[11095]: modified /vm-disks/hdd/XXXH.qcow2, saved in 334.554s (339.774 MiB added)
Jul 24 09:53:00 kvm1807 restic[11095]: modified /vm-disks/ssd/XXXI.qcow2, saved in 25.250s (181.909 MiB added)
Jul 24 09:53:09 kvm1807 restic[11095]: modified /vm-disks/ssd/XXXJ.qcow2, saved in 84.716s (212.253 MiB added)
Jul 24 09:53:16 kvm1807 restic[11095]: modified /vm-disks/ssd/XXXK.qcow2, saved in 35.427s (352.978 MiB added)
Jul 24 09:53:18 kvm1807 restic[11095]: Files: 0 new, 10 changed, 1 unmodified
Jul 24 09:53:18 kvm1807 restic[11095]: Dirs: 0 new, 0 changed, 0 unmodified
Jul 24 09:53:18 kvm1807 restic[11095]: Data Blobs: 3434 new
Jul 24 09:53:18 kvm1807 restic[11095]: Tree Blobs: 1 new
Jul 24 09:53:18 kvm1807 restic[11095]: Added: 2.193 GiB
Jul 24 09:53:18 kvm1807 restic[11095]: processed 11 files, 267.013 GiB in 6:13
Jul 24 09:53:18 kvm1807 restic[11095]: snapshot 086c6a5d saved
Jul 24 09:53:18 kvm1807 systemd[1]: restic@vmdisks.service: control process exited, code=exited status=203
Jul 24 09:53:18 kvm1807 systemd[1]: Failed to start restic "vmdisks" backup.
Jul 24 09:53:18 kvm1807 systemd[1]: Unit restic@vmdisks.service entered failed state.
Jul 24 09:53:18 kvm1807 systemd[1]: restic@vmdisks.service failed.