Compresssed data despite RESTIC_COMPRESSION=off


Im currently updating from 0.13.2 to 0.16.0. We decided that we don’t want to use compression, as space is not a concern.

I upgraded a test repository to the v2 format, as well as deleted the repository to create a clean state.

Compression is deactivated via ENV variable. Printing the env variable within the wrapper shows it’s set to “off” (all variables are sourced and exported).

As there is no snapshot to prune, there is also no repacking involved. In addition, i explicitly set ‘–compression off’ for the backup command. This is how my stats look after the initial backup:

restic-wrapper stats --mode raw-data
repository 052dab90 opened (version 2, compression level off)
Stats in raw-data mode:
     Snapshots processed:  1
        Total Blob Count:  89411
 Total Uncompressed Size:  4.782 GiB
              Total Size:  4.757 GiB
    Compression Progress:  0.70%
       Compression Ratio:  4.29x
Compression Space Saving:  0.54%

On a different system, i created two additional snapshots, stats are as followed.

restic-wrapper stats --mode raw-data
repository 2def7b31 opened (version 2, compression level off)
Stats in raw-data mode:
     Snapshots processed:  9
        Total Blob Count:  283164
 Total Uncompressed Size:  27.774 GiB
              Total Size:  27.665 GiB
    Compression Progress:  0.40%
       Compression Ratio:  788.00x
Compression Space Saving:  0.40%

As you can see, the env variable is processed by the restic, and despite setting compression off, i end up having compressed data in my repository. Backup output confirms my observations.

Log excerpt from first stats overview:

2023-08-17T13:10:20+02:00 Starting backup
no parent snapshot found, will read all files

Files:       81040 new,     0 changed,     0 unmodified
Dirs:        13251 new,     0 changed,     0 unmodified
Added to the repository: 4.780 GiB (4.760 GiB stored)

processed 81040 files, 5.090 GiB in 1:17
snapshot 391816c6 saved

Log excerpt from second stats overview:

2023-08-17T12:21:34+02:00 Starting backup
using parent snapshot 13044e91

Files:          10 new,    47 changed, 250639 unmodified
Dirs:            8 new,    59 changed, 28376 unmodified
Added to the repository: 72.042 MiB (15.755 MiB stored)

processed 250696 files, 467.849 GiB in 6:25
snapshot ec9a4fbd saved

I tried 0.16.0 and 0.15.2 (Source: $version_linux_amd64.bz2 from Github), with no difference in behavior. Any idea on how to advance from here, any debugging advice?

The stat command counts both the size of the files’ contents and the directory metadata.
The RESTIC_COMPRESSION=off setting only applies for the files’ contents but not the directory metadata. The latter is always compressed with no option to disable that. The reason for that design decision is that the metadata is usually highly compressible which results in a much smaller local cache size.

Thanks for getting back, Michael.
I’d consider to add this information to the documentation.