Restic-temp-files in repository

pille · June 13, 2023, 10:08am

i’ve noticed lots of files in my repositories index and data subdirectories that are not covered by the every-file-is-named-by-its-content-hash-policy.

the naming scheme is the following:
32BYTEHEX-restic-temp-16BYTEHEX

one example would be:
36de2e7af3b41cf056f8a8da2f790069686157a788dea4d8203ac83fc33ecd81-restic-temp-6cf330d17ecc94ecf141d7bd45732b5a
(not matching SHA256 of content)

the repository was created about 2 years ago (v1).
those files appear since about 1 year, according to their timestamps.
the repository is periodically pruned, but those files stay.

what are they about and in case they are really unneeded (“temp”), why doesn’t a prune get rid o fthem?

pille · June 13, 2023, 10:17am

to partly answer my question:

github.com

restic/restic/blob/237f32c651d4924242783a02c9a52b09ccc2dc4f/internal/backend/sftp/sftp.go#L305


      
          	return hex.EncodeToString(nonce[:])
          }
          
          
// Save stores data in the backend at the handle.
          func (r *SFTP) Save(_ context.Context, h restic.Handle, rd restic.RewindReader) error {
          	if err := r.clientError(); err != nil {
          		return err
          	}
          
          
	filename := r.Filename(h)
          	tmpFilename := filename + "-restic-temp-" + tempSuffix()
          	dirname := r.Dirname(h)
          
          
	// create new file
          	f, err := r.c.OpenFile(tmpFilename, os.O_CREATE|os.O_EXCL|os.O_WRONLY)
          
          
	if r.IsNotExist(err) {
          		// error is caused by a missing directory, try to create it
          		mkdirErr := r.c.MkdirAll(r.Dirname(h))
          		if mkdirErr != nil {
          			debug.Log("error creating dir %v: %v", r.Dirname(h), mkdirErr)

they are part of the SFTP backend upload procedure.
since their hashes don’t match and they were not removed, i guess the connection dropped during upload.

the question why they did not get pruned, still remains.
Removing backup snapshots — restic 0.15.2 documentation.

MichaelEischer · June 16, 2023, 5:09pm

Simple, there’s no cleanup implemented for these files so far. Feel free to open an issue . How many files are we talking about?

pille · June 17, 2023, 10:55am

there were about 50-100 of those files.
i’ve opened an issue, as i expected a prune to remove all superfluous files: prune does not remove all superfluous files · Issue #4372 · restic/restic · GitHub