I just mounted some backups using fuse into my home directory, and woah was this a bad idea. Tracker, the program that makes files full-text searchable in linux, has decided to start scanning the mounted directory. I imagine that behind the scenes it’s currently having a wild time scanning files and directories because my fans are blazing any my CPUs are pegged.
There’s some documentation around about how to tell Tracker to leave a directory alone. I’m not sure which of the docs are outdated, and I suspect that we don’t want to have to tell every filesystem indexer to leave things alone. Maybe this should go in the docs?
Just thought I’d mention this since it’s a bit of a curious thing that’s happening. Only solution I’ve got is to umount the fuse as soon as possilbe.
Is this question in any way related to restic the backup software?
Yes. Restic has an option to mount backups as a fuse mount point. This is a really cool feature, but my point was that people should be warned that if they do this on, say, a Linux box, all kinds of things can come and start traversing that mount point.
I don’t know if there’s a way to avoid this, but the example that I ran into was Tracker, the daemon on Ubuntu that makes all the files full-text searchable. It came along, found the fuse mountpoint and started downloading and extracting files to make them full-text searchable. That’s probably bad since it:
Runs up your CPU
Downloads files you don’t intend to download
Makes those files searchable, which you don’t really want
Can run up your bill with services like backblaze, that charge by the bandwidth
There’s also the locate command that might come and start traversing these directories if they’re mounted. That’s bad too, and even more common than Tracker, I’d imagine.
Gotcha! But still it doesn’t seem to be about restic but a different program.
Isn’t that the point of mounting things via fuse?
Don’t get me wrong I can see and understand what you’re concerned about and what your problem is.
As far as I can tell, this topic has not come up before. So it might be an edge case only a small number or restic users would run in to.
And even then, it’s something not related to restic.
IMHO your best bet is to see if you can exclude the restic fuse mount point for this program.
Well, it’s the combination of them. Maybe there’s a way restic can tell other programs to stay out, for example, by including a dot file in the root of the directory when it mounts them. Maybe restic can include a warning about this in the mount documentation. Maybe the other programs can be taught about restic and can identify restic mounts and ignore them. I don’t know, but one side or the other could do something about this, and I suspect both will blame the other.
As far as I can tell, this topic has not come up before. So it might be an edge case only a small number or restic users would run in to.
Hard to say. I’d bet it’s happening more than we realize and that most users just don’t notice. Meanwhile their CPU, bandwidth, b2 storage fees, and etc are being used. What OS are you on? Does it have a file search tool? Does that tool include mounted directories? I betcha it does…
IMHO your best bet is to see if you can exclude the restic fuse mount point for this program.
Right, but I wonder if that’s something restic could do on its own or, if that’s not possible, at least warn users about. I guess this conversation means there’s no interest in it. That’s fine, but I’m pretty sure others will have some form of issue with this whether they realize it or not.
I don’t see this happening for multiple reasons. Let’s say we do this; how can we verify that this works reliably? Are there other tools out there that work differently and need different restic options?
Restic can’t accomodate all other programs in regards of the other programs functionality.
As far as I can tell from googleing, there is an option in that program you mentioned to exclude certain folders.
I would suggest that you add this exclusion I mentioned earlier - excluding the path to the fuse mount point of the restic repository.
I haven’t seen other programs that utilize fuse mounts as well print a “warning” to the user that files are not searchable by other programs. To me at least, this is obvious. If you mount a disk and it has files on it, and you have a program that does indexing then it’s inevitable that this program will eventually start indexing that new disk with all its files if it is set up that way.
Not sure if anyone else wants to pitch in here - as this is not something I can even decide or have say over - but I don’t see restic changing in that regard, sorry.
Maybe. Should we do this for every program that might recursively search a directory structure? This is an untenable solution – we can’t control what other programs might pop up and crawl directories. We might not even be able to tell other programs to stay out.
A side effect of making a virtual structure that looks like a local filesystem is just that: it looks like a local filesystem. You’d have this same problem with sshfs or s3fs or rclone mount or…
Probably the most we could do is make a documentation note about it in this section.
OK, so a few things. First, we can make it so that the updatedb command doesn’t come crawling. In it’s config it has a section called PRUNEFS. Here’s how it’s man page describes it:
A whitespace-separated list of file system types (as used in /etc/mtab ) which should not be scanned by updatedb (8) .
And if you go looking in there, you’ll actually see a handful of other fuse file systems (at least on Ubuntu):
As for other programs, the docs say to mount it under /mnt. That’s good. But if you mount under somewhere else, you have these kinds of issues. Why not mention this to the user? Restic could pretty easily emit a warning when users mount things in a risky place. Something like:
Warning: You’ve mounted this at /home/you/restic. This location may be scanned and indexed by various programs on your operating system, such as updatedb, Tracker, or others. We recommend mounting under /mnt