Restored folder size is much smaller than expected

Getting Help
1

Hello,

I am trying to restore a folder from a backup snapshot.

This is the snapshot from “restic snapshots”:
54cb4086 2024-10-30 05:00:01 xxxx xxxx
/xxxx
/xxxx/admin
/data
/storage
/var/lib/docker-data

Backup was performed using this version:
restic 0.16.0 compiled with go1.20.6 on linux/amd64

Restore was tried on the machine hosting the backup. I tried with:
restic 0.16.2 compiled with go1.21.3 on linux/amd64

and also:
restic 0.17.1 compiled with go1.23.1 on linux/amd64

The only ENV variables set are the RESTIC_REPOSITORY (which is a local folder on this machine) and RESTIC_PASSWORD.

ISSUE:
I am trying to restore a folder which contains a neo4j backup by issuing the following command:
restic restore 54cb4086:/data/graph/graph_db_backup/2024_1027/neo4j --target /mnt/backups/test-restores/graph/xxxx-data/2024_1027/neo4j
repository 4a9693bb opened (version 2, compression level auto)
[0:59] 100.00% 325 / 325 index files loaded
restoring snapshot 54cb4086 of [/xxxx /xxxx/admin /var/lib/docker-data /data /storage] at 2024-10-30 05:00:01.169552618 -0400 EDT by root@xxxx to /mnt/backups/test-restores/graph/xxxx-data/2024_1027/neo4j
Summary: Restored 188 files/dirs (80.350 GiB) in 10:33

When I check the restored folder size it is much smaller than expected (80.350 GiB)
du -sh *
31G neo4j

Am I doing anything wrong? The 80G size is the size I expect.

Thank you

1 Like
2

To be sure, are there hidden files directly in that folder? du -sh * does not count them. What does du -sh /mnt/backups/test-restores/graph/xxxx-data/2024_1027/neo4j say?

1 Like
3

The file size is reported the same:
du -sh /mnt/backups/test-restores/graph/xxxx-data/2024_1027/neo4j/
31G /mnt/backups/test-restores/graph/xxxx-data/2024_1027/neo4j/

Also if I check the size against a restic mount the larger expected size is reported.
./restic_0.17.1_linux_amd64 mount /mnt/restic_mount/
repository 4a9693bb opened (version 2, compression level auto)
[1:00] 100.00% 323 / 323 index files loaded
Now serving the repository at /mnt/restic_mount/
Use another terminal or tool to browse the contents of this folder.
When finished, quit with Ctrl-c here or umount the mountpoint.

from another window:
du -sh /mnt/restic_mount/ids/54cb4086/data/graph/graph_db_backup/2024_1027/neo4j/
81G /mnt/restic_mount/ids/54cb4086/data/graph/graph_db_backup/2024_1027/neo4j/

I also tried copying the folder from the mount (which took much longer as expected) but the size was still the 31G.

4

Is the filesystem to which you restored the backup maybe compressed? Or does the snapshot contain hardlinks?

5

Bingo :wink:

Turns out the filesystem I am restoring to is zfs with compression :wink: I inherited this from someone else and am not familiar with zfs. I restored to another system using xfs filesystem and the size is as expected. Thanks for the help - much appreciated.

6

That’s a bit odd – I don’t think that du is aware of compression. I use btrfs with compression quite a bit, and du only looks at the size of the file reported on the inode, which is the actual decompressed length. I suspect that there is some other explanation.

7

I have to admit I don’t have much experience with compressed file systems, however it does seem to be related as du and ls report different sizes.
Here is an example on the largest file in the folder.

du -h neostore.propertystore.db

15G neostore.propertystore.db

ls -lh neostore.propertystore.db

-rw-r–r-- 1 7474 7474 48G Oct 30 02:45 neostore.propertystore.db

8

It is perfectly correct what it does. Maybe du is not aware of encryption per se but indirectly for sure as it simply shows how many sectors on disk are occupied by data.

du does not display files nor dirs sizes but Disk Usage… hence it is called du:slight_smile:

It can be used to show real size with extra flag. As per man du:

-A Display the apparent size instead of the disk usage. This can be helpful when operating on compressed volumes or sparse files.

1 Like
9

Yes, ‘du -sh’ shows 31G but ‘du -sh --apparent-size’ shows 81G. Thanks for all the input, always something new to learn :slight_smile:

2 Likes
10

I wonder if btrfs doesn’t present compression information in a way that du understands but zfs does, then, because du on compressed btrfs files always shows the uncompressed size. You have to use the compsize tool to see actual disk usage regarding compression on btrfs.

11

I think BTRFS is massively work in progress compared to ZFS. Missing underlying disks block du is probably not top priority given that it still has to fix such core features like RAID 5 and 6.

We all have learnt something new in this thread:) very good:)