Help understanding the --force option

taikobo · May 15, 2025, 4:37pm

I read through all the documents and did not find an answer, any help is much appreciated!

It says the --force command will ignore the parent snapshot and rescan all files.

Then what? How does restic decide whether or not the files are changed and needed to be uploaded? Based on my observation, files which haven’t changed will not be uploaded. Does restic keep a sha256 of all source files from the previous backup, and then it calculates the current sha256 and compares it? Then it contradicts the information I gathered from the github issues, it seems that only checksum of pack files are calculated and kept, calculating checksum for source files is deemed too resource intensive and not performed.

I am confused as to what happens.

shd2h · May 15, 2025, 5:13pm

Then what? How does restic decide whether or not the files are changed and needed to be uploaded?

It kinda doesn’t need to. You used --force, so when the backup command is run, restic “chunks” all of the incoming data files, splitting them up using content defined chunking.
This process produces a bunch of data blobs, which are assembled into pack files for uploading to the repository.
These blobs are what is compared to the existing data in the repository.
If the data doesn’t already exist in the repository, it gets uploaded. If it does, then no upload happens.

Put another way, the entire file is not compared to what is stored in the repository, but the contents of the file (after being split into pieces) is compared.

On the other hand, if you didn’t use --force, and the ctime, mtime and inode of the file matched the information stored in the previous backup, restic would not “chunk” the file, and so no comparison of the actual data would take place. This is because the file metadata indicates the file content has not changed since the last backup.

The above is my understanding anyway, I’m sure if I’ve made any mistakes, other more knowledgeable users will correct me

taikobo · May 15, 2025, 5:19pm

Thanks! I have been reading through a lotf and your explanation aligns with my understanding.

Reading through the design document:

github.com/restic/restic

doc/design.rst

master


Terminology
===========

This section introduces terminology used in this document.

*Repository*: All data produced during a backup is sent to and stored in
a repository in a structured form, for example in a file system
hierarchy with several subdirectories. A repository implementation must
be able to fulfill a number of operations, e.g. list the contents.

*Blob*: A Blob combines a number of data bytes with identifying
information like the SHA-256 hash of the data and its length.

*Pack*: A Pack combines one or more Blobs, e.g. in a single file.

*Snapshot*: A Snapshot stands for the state of a file or directory that
has been backed up at some point in time. The state here means the
content and meta data like the name and modification time for the file
or the directory and its contents.

This file has been truncated. show original

It seems that checksum is performed on blob files. So the --force option recalculates the checksum of blob files and compares it with existing blob checksum.