Problem with fsfreeze

I have written a wrapper restic-xbackup which calls fsfreeze before restic backup to ensure that there is no writing to the filesystem as long as the backup is running.

I know restic is using TMPDIR, so I set:

# echo $TMPDIR 
/run

# df /run
Filesystem     1K-blocks  Used Available Use% Mounted on
tmpfs            1048576  1180   1047396   1% /run

But restic backup still runs in a deadlock, I assume it tries to write something to the / filesystem:

# fsfreeze -f /

# restic backup -v -v -x --exclude-file /root/.restic/exclude --tag _ / 
open repository

Here restic stops, deadlock.

What can it be?

Writing to cache? If restic runs on the machine that holds the root filesystem that you try to back up, how did you define the restic cache folder?
https://restic.readthedocs.io/en/latest/manual_rest.html#caching

1 Like

Ahh… cache directory!
Option --no-cache solves my problem with fsfreeze!
THANKS!

Update: --no-cache makes restic very slow

Meanwhile I am using TMPDIR=/run which is a tmpfs filesystem not interacted with fsfreeze

1 Like

I have still problems with fsfreeze: restic backup is in a deadlock.

I use:

export TMPDIR=/run
mount -o remount,size=1G /run
fsfreeze -f /
restic backup -v -v -x --exclude-file /root/.restic/exclude --tag _ --cache-dir /run/restic /

/run/restic is used, I see:

root@obertux:/run/restic# ll
drwx------ root root  - 2024-06-19 16:06:28 abe9456226872758e26577a8b4efd183b463f392ea5e7392b2d7eef4b05fb095
-rw-r--r-- root root 44 2024-06-19 16:06:28 CACHEDIR.TAG

but restic hangs after:

open repository
lock repository
using parent snapshot 1a31a786
load index files
start scan on [/]
start backup on [/]
scan finished in 2.998s: 136469 files, 7.342 GiB

I have to enter “fsfreeze -u /” (in a second window).

restic even hangs when I use

restic backup -v -v -x --exclude-file /root/.restic/exclude --tag _ --no-cache /

So, there must be a write of something other, too - and the freezed / filesystem suspends restic.
==> deadlock

For debugging I run:

strace restic backup -v -v -x --exclude-file /root/.restic/exclude --tag _ --no-cache / &>/run/restic.log

While restic was runnig, I run “tail -f /run/restic.log”:

write(1, "unchanged /boot/grub/i386-pc/pxe"..., 42) = 42
futex(0x1b6c948, FUTEX_WAIT_PRIVATE, 0, NULL) = 0
llistxattr("/boot/grub/i386-pc/raid6rec.mod", 0x1b9c9c8, 0) = 0
newfstatat(AT_FDCWD, "/boot/grub/i386-pc/random.mod", {st_mode=S_IFREG|0644, st_size=3708, ...}, AT_SYMLINK_NOFOLLOW) = 0
futex(0x1b6c948, FUTEX_WAIT_PRIVATE, 0, NULLscan finished in 3.806s: 136477 files, 7.342 GiB

At this point there was the deadlock.
After “fsfreeze -u /” I saw:

Files:           0 new,     0 changed, 136477 unmodified
Dirs:            0 new,     1 changed, 18158 unmodified
Data Blobs:      0 new
Tree Blobs:      1 new
Added to the repository: 11.969 KiB (2.403 KiB stored)

processed 136477 files, 7.342 GiB in 1:50
snapshot eecae221 saved
) = ?
+++ exited with 0 +++

I cannot see any problematic writing to the freezed / filesystem.

With try&error I have detected why restic backup runs in a deadlock with fsfreeze:

# export TMPDIR=/run
# fsfreeze -f /tmp
# restic backup -v --exclude-file /root/.restic/exclude --tag _tmp --no-cache /tmp
SG_TENANT=tik-sas-linux
RESTIC_REPOSITORY=s3:s3.tik.uni-stuttgart.de/tik-sas-linux.obertux.restic
open repository
repository abe94562 opened (version 2, compression level auto)
lock repository
using parent snapshot f8207fc1
load index files
[0:00] 100.00%  2 / 2 index files loaded
start scan on [/tmp]
start backup on [/tmp]

Here restic hangs again!

This happens only when I use /tmp as the backup argument, no problems with any other directory!

==> restic tries to write something to /tmp, ignoring the TMPDIR environment variable!

Because my /tmp is a subdirectory of / and not a tmpfs filesystem. it comes to the deadlock.

I filed in a bug report: github