Both the source folder and the repo are stored on MergerFS over ext4.
If I believe Restic detecting (non-existing?) changes result in long backup time, MergerFS does not preserve inodes, so I guess I have my culprit.
Using --ignore-inode should do the trick, right?