Backup, but keep deleted files on the cloud

Hello,

I plan to use restic to backup my files to B2, and would like to ask whether it is possible to do that, but when I delete files locally, these files should still be kept on the cloud. In other words, I don’t want a “mirror” scheme here. Local storage is finite but cloud storage is not, and is cheap nowadays, so I would like to keep everything there.

Is there some special configuration to apply to Restic in order to achieve this?

Thank you very much in advance

This is exactly what restic is all about. You back up your stuff, and each time a snapshot is created to represent what you backed up at that point in time. When you delete files locally it only affects backups you make later on - the previous snapshots will still keep the data that’s in them. They will only be cleaned out when you use the forget command along with prune.

So no, you don’t need any special configuration to do this. I recommend you read a bit in the documentation at https://restic.readthedocs.io/en/latest/ to get a more complete understanding of how it works.

1 Like

Thanks for the quick reply @rawtaz!

So, if I have fileA locally, take a backup that creates snapshotA, then delete fileA from my computer and take another backup snapshotB, then fileA will only exist in snapshotA, as long as I haven’t used “forget” along with “prune”?

What if I want to recover it? I have to manually search all past snapshots, which could be hundreds?

Exactly how you describe it, yes.

If you want to restore files from your backups, you can do it in different ways. If you can install FUSE on your system, you can mount the repository and get a regular directory tree with all your snapshots and their contents, so you can copy it like any other filesystem. Another way is to use the restore command, see the manual.

See the “Restore using mount” and “Restoring from a snapshot” headings at: https://restic.readthedocs.io/en/latest/050_restore.html

Well, normally you have an idea about when you last had the file, so you can simply go by that when looking through your snapshots. If you don’t, you can do a similar thing to git’s bisect command, which pretty quickly narrows it down to the one where you last had the file you’re looking for.

I should also add that forget is the command that removes the snapshot from the history. The prune command just cleans out the data that was unique to that snapshot.

What I mean to highlight is that I would not count on it being possible to retrieve a snapshot/files when you have run forget but did not yet run prune. It might be doable with some magic touch by @fd0 but it’s certainly not intended to be used that way. So consider forget the thing you use to remove the backed up files/snapshots from your repo, without return :slight_smile:

I see! But then again, I have to know what I’m looking for. In case I’m looking for some photos I might haven taken about a year ago when I went to Spain, but don’t remember any filenames, or dir names, then the restic approach might not be very appropriate. Maybe rclone would be a better fit.

I think for long-term, “cold storage” I would use rclone, and have restic backup (well, “mirror”) my home folder (which includes firefox/thunderbird profiles, etc), for which history beyond a week or so is not important.

Thanks for the clarifications @rawtaz ! :slight_smile:

Let me be very very clear; restic and rclone are two different types of software. Please do not consider them the same, and especially do not consider rclone a backup software that can do the things restic can on that topic. Rclone is great software, but not purposed for backing up in the sense that restic is designed to do for you.

By what you have said so far, I highly recommend that you use restic for your backups.

Yes, if you have no idea what you are looking for, you will have to look around. But how is that different from a mirror you made with rclone? You have equally little knowledge of what you are looking for with that too, so there’s effectively no difference.

Again, the mount feature of restic is superb - using it, you can mount your backup repository and browse the snapshots and files in it like they were on your local disk (just with some added latency). So you could e.g. go into snapshot by snapshot and look around in your folders, and peek at the various images, like they were on your hard drive. I don’t know how you could possibly get closer to finding what you’re looking for, except with additional tagging and metadata of course.

Tagging and metadata is something restic supports as well - you could tag your backups with whatever makes sense to you, e.g. tag the snapshot you make right after having stored the vacation photos from Spain in your hard drive with just that, “spain”. Then you’ll be able to locate that particular snapshot in the list of snapshots. You can tag when you back up, or you can edit tags later.

I really think restic is what you need to keep your files backed up in a good way. To be honest I think your concerns are more due to not entirely understanding the features of restic, rather than restic not being the right tool for the job. And I’m saying all this without bias.

1 Like

Great answer @rawtaz! Thanks for being so patient and exhaustive!

One of the most basic methods of backup is having an external hard drive, which you plug in every now and then, and copy to it everything valuable. I might choose to overwrite my firefox profile folder (bookmarks, saved passwords etc), and I might choose to add the “Spain” under the Videos root folder of my external drive. After I delete the videos off my local drive to free up some space, I still need to have them saved in the external drive. And when I need to find them, I’ll look under Videos.

I’m not sure what this is called. I don’t think it’s called “mirroring”, nor “synchronizing”, nor just “incremental backup”. But with Restic, I won’t have all of my valuable files under one tree structure, they will be spread between snapshots, because of the -otherwise very useful- versioning feature. I will have to potentially mount several snapshots before finding what I’m looking for.

On the other hand, I think rclone (rsync even) can do just this: find what’s new locally and “add it” to a single target remotely.

Granted that tags help here, although one might end up reproducing the folder names into tags, just so they can navigate their snapshots.

Maybe I’m missing something here, but I’m only trying to make an informed decision! And I thank you again for this!

Sounds to me like what you are describing is simply another disk to store your stuff on (having them spread out over multiple disks). You want the cloud thingie to be a second hard drive, effectively.

It’s of course not backup, since if you delete the file(s) from where they’re stored, they’ll be lost.

So that’s the basic aspect of this. Then we have rclone, which just becomes a tool for you to transfer files to/from that second “hard drive”, e.g. instead of just copying the files to and from your hard drive manually in some file browser.

Yes, you are right that rclone can do a “one way sync”, such that it copies whatever doesn’t exist on one side from the other side. It has quite a number of features :slight_smile:

No worries about your questions, and welcome to the forum!

I’d like to add another thought: With rclone/rsync, you won’t notice when your hard drive stops being reliable. With restic on the other hand, you can run restic check and it’ll tell you when the data returned from the disc has been modified.

3 Likes

A feature that many will not appreciate until its too late. :stuck_out_tongue:

11 posts were split to a new topic: Unified view of latest versions of all files in all snapshots

2 posts were merged into an existing topic: Unified view of latest versions of all files in all snapshots