Can we see snapshot sizes?


#1

I have been trying to go through all the documentation (it seems to be scattered and really incomplete), but I cannot seem to find a way to see how much storage my snapshots are taking. I can do this in borg (which I have been testing for a few days), and was hoping to utilize restic instead of a borg+rclone combo. Even the built in help for some of the commands is cryptic and incomprehensible.


#2

Hi,

It would be great if you could be more specific regarding what is lacking in the documentation, and especially the commands help - what is cryptic and incomprehensible? If we don’t know what you’re reacting on, it’s hard to improve it.

Thanks!


#3

I also want to see snapshot ‘size’ (or something related to size)

But at the same time I can’t define what does ‘size’ means… It will be cool to see followed info somewhere:

  1. Amount of data that is referenced by snapshot (disk space needed to do ‘restic restore ID’)
  2. Amount of data added to repository with this snapshot
  3. Amount of data that will be removed from repository in case of ‘forget & prune’ for this particular snapshot.

#2 probably can be calculated during backup, but it’ll be outdated on first ‘forget+prune’ cycle.

Maybe some sort of ‘restic du’ command is needed. Also #1 can be calculated using ‘restic mount’ and then ‘du’ on fuse mount point.


#4

This is what I was thinking of:

It would be great if you could be more specific regarding what is lacking in the documentation

When I look at the documentation between Borg and Restic, it appears that Restic doesn’t offer a lot of the same functionality as Borg. To me, Restic seems like a good blend of Borg (dedup and archive) and rclone (cloud storage connection)

what is cryptic and incomprehensible?

If I issue restic --help I get

root@backup1:/ # restic --help

restic is a backup program which allows saving multiple revisions of files and
directories in an encrypted repository stored on different backends.

Usage:
  restic [command]

Available Commands:
  autocomplete  generate shell autocompletion script
  backup        create a new backup of files and/or directories
  cat           print internal objects to stdout
  check         check the repository for errors
  dump          dump data structures
  find          find a file or directory
  forget        forget removes snapshots from the repository
  help          Help about any command
  init          initialize a new repository
  key           manage keys (passwords)
  list          list objects in the repository
  ls            list files in a snapshot
  migrate       apply migrations
  mount         mount the repository
  prune         remove unneeded data from the repository
  rebuild-index build a new index file
  restore       extract the data from a snapshot
  snapshots     list all snapshots
  tag           modifies tags on snapshots
  unlock        remove locks other processes created
  version       print version information

Flags:
  -h, --help                   help for restic
      --json                   set output mode to JSON for commands that support it
      --no-lock                do not lock the repo, this allows some operations on read-only repos
  -o, --option key=value       set extended option (key=value, can be specified multiple times)
  -p, --password-file string   read the repository password from a file (default: $RESTIC_PASSWORD_FILE)
  -q, --quiet                  do not output comprehensive progress report
  -r, --repo string            repository to backup to or restore from (default: $RESTIC_REPOSITORY)

Use "restic [command] --help" for more information about a command.

yet, if I say restic list I get

root@backup1:/ # restic list
Fatal: type not specified

I gave it an invalid command, but instead of a --help screen, I see this error message. Part of that is me confusing borg commands and Restic commands, but on most any other program if I give it an invalid command it would list the possible commands and options for me (man page style)

Now if I issue: restic list --help:

root@backup1:/ # restic list --help

The "list" command allows listing objects in the repository based on type.

Usage:
  restic list [blobs|packs|index|snapshots|keys|locks] [flags]

Flags:
  -h, --help   help for list

Global Flags:
      --json                   set output mode to JSON for commands that support it
      --no-lock                do not lock the repo, this allows some operations on read-only repos
  -o, --option key=value       set extended option (key=value, can be specified multiple times)
  -p, --password-file string   read the repository password from a file (default: $RESTIC_PASSWORD_FILE)
  -q, --quiet                  do not output comprehensive progress report
  -r, --repo string            repository to backup to or restore from (default: $RESTIC_REPOSITORY)

OK, but if I issue: restic list snapshots:

root@backup1:/ # restic list snapshots
Fatal: Please specify repository location (-r)

This seems to contradict what the documentation says:

restic -r /tmp/backup snapshots

so to me, the help screen and the documentation do not match. I don’t know what commands I am supposed to use (are there more commands not listed in documentation?). I have become quite confused by what commands I am supposed to issue and what the expected output should be. I only found this because I was trying to see if I could get archive/snapshot sizes to display.

Sorry for my ramblings, I was confused last night, and it still doesn’t make a whole lot of sense to me today.


#6

This may be a nice improvement indeed.

But just tested with 2 first basics commands that came in my mind “ls -z” and “cat -z” and this only report “invalid option – ‘z’” …

ok, so here 109.40GB is the total space used on the remote storage ? And the value you were looking for with restic ?
I have not seen that option with restic (but I’m new). This may be a good improvement indeed.


#7

The documentation looks ok to me… but I think you are mixing 2 commands

  • restic snapshots (to list snapshots)
  • restic list (to list objects)
    restic list [blobs|packs|index|snapshots|keys|locks] [flags]
    snapshots is one of the objects

#8

Hey, thanks for the feedback.

That’s in my opinion a philosophical issue. I’m a bit annoyed sometimes by error messages that are printed at the top of a very long manpage/options list, so I need to scroll up and find the error. I actually prefer printing the error, and having the users (when they need help) issue restic cmd --help.

I agree that the documentation and especially the error messages can be improved (a lot). Part of the problem is that you stumbled upon the list command, which is actually used to list things in the repository on a very low level, it is basically not meant for end-users to use. The snapshots command is used to display information about snapshots in the repo.

The manual starts by explaining how to display help for commands and how to initialize a repository. Among others, this also explains the environment variable RESTIC_REPOSITORY, here: https://restic.readthedocs.io/en/latest/manual.html#initialize-a-repository

If you have an idea on how to improve the documentation, please let us know!

Adding more statistics is (among others) tracked in https://github.com/restic/restic/issues/693.


#9

Actually, I want to see the ‘DEDUPLICATED’ size because that is what I will transfer to my B2 bucket

What? It literally says that I can use restic list snapshots right there…as well as blobs, packs, index, keys, and locks.

I am testing restic, borg, rclone, and any other deduplication backup/archive software I can right now. I have been using Crashplan for the last 6+ years and now they are cancelling all their home plans, so I have to find a replacement. I like the idea of something like restic or borg+rclone, with a Backblaze B2 storage, but I need to be cognizant of how much space I am taking up before I transfer it over my horrible internet connection, and I also need to know how much my backups will cost me monthly. I have several test repos that I am playing around with and I do not want to muck with ENV variables until I decide what option I am going to go with.

With Crashplan going away, I think we will see an uptick in software like Restic/etc being used, and I think most people will be interested in knowing how much space their backups are taking.


#10

And you can do that. As I pointed out above, restic list is a low-level command. restic list snapshots will print a list of snapshot IDs, nothing more. On the other hand, restic snapshots prints a nice table with human-readable information. That’s also described in the manual here: https://restic.readthedocs.io/en/stable/manual.html#list-all-snapshots


#11

Right, I have seen the documentation. This is where I am confused. The documentation has commands backwards from how the man page describes the commands.

I guess what I am looking for doesn’t exist in Restic, and perhaps Restic is the wrong tool for me. From the onset, I saw it to be a replacement for borg + rclone combination, but I feel like I cannot get the information I need from it. The documentation page has a list of commands available, but doesn’t go into detail on how to use those commands or what they are for, and the man page has a different command structure than the documentation page. I have had no issues with learning how to use Borg, so I thought I wouldn’t have any issues with Restic, but I am just not understanding the layouts here.


#12

Let me try to explain my frustration a little better.

https://restic.readthedocs.io/en/stable/manual.html#usage-help

From there, you can see the usage states:
restic [command]
then there is a list of commands and a list of flags, with no indication of when or where to use those flags or in what order (if it matters) in how those flags can be used.

In the list of commands, many are ‘undocumented’ on the documentation site, and the only way to see what they do is to use the help in the command line, with no indication of examples or structure.

When I read https://borgbackup.readthedocs.io/en/stable/usage.html#general I feel like the documentation is structured in a way to explain nearly EVERYTHING the program does without having to switch back and forth from a documentation page to a terminal.

It’s a layout and understanding issue on my part. I think you guys understand it more because you work with it all the time and know the ins and outs, where idiots like me are fumbling around trying to figure out how to utilize this program best.