NPBackup: Cross platform restic based backup solution, batteries and GUI included

Community
1

Hello all,

After having benchmarked a couple of deduplicating open source backup programs here, I decided that restic was a good choice for my needs, backing up both servers and end user computers.

I’ve invested some time into building my restic ecosystem, and ended up building an all in one restic based solution which supports both Linux and Windows.

Features:

  • Prometheus monitoring support for servers (pull) and laptops (push), including a Grafana Dashboard
  • Full end user GUI with backup content viewer and restore function
  • CLI for server scheduled tasks
  • Configuration via YAML file and/or GUI
  • Additional security (repo URI and passwords are AES-256 encrypted in configuration file)
  • Big ready to use exclusion lists for Linux and Windows
  • First class Windows support (filter out reparse files, no-VSS fallback, pre-built binaries without dependencies)
  • Backup process and IO priority settings
  • File restoration direct to disk without the need of intermediate zip files

License
After having run this software for some time for my internal needs, I begun thinking that all this was made possible by the restic authors, which dedicate time into that awesome project, and keep it open source. So here I am today releasing my software under the GPLv3 license. I will continue to make it evolve (see TODO.md) file, and I would be pleased to get community feedback.

Downloads
You can find the github repository at https://github.com/netinvent/npbackup

Internals

I use this software mainly to backup servers with virtual machines (I have another software that handles the pre and post-backup freeze/thaw operations for the virtual disks).

Nevertheless, I have some special usecases, one of which needs customer laptops to be backed up.
Of course, since they’re never powered on at the same time, using a schedule is not the right idea.
Also, since they travel, the laptops are never connected to the same network.

I resolved this by allowing NPBackup to connect to it’s repo every 15 minutes via a scheduled task, and launch backup if no recent snapshot already exists.
The monitoring of laptops can push the metrics directly to a prometheus push gateway, which also resolves the different network issues.

Other particularities are that NPBackup properly filters out cloud files (Onedrive, Nextcloud…) which are not stored on the computer, and hence cannot be backed up on Windows.
This is the default behavior as of current version, but will become configurable so more usecases are covered.

A last thing is that the backup priority is configurable, for CPU but also IO, in order to not slow down other programs while backups take place.

Screenshots

Yes, a plain GUI, which supports direct interaction. Have a look at the other screenshots on the github page.

Make it reliable, only then, make it beautiful

image
image619×507 23.8 KB

Misc
Yet another restic GUI ?
Yes, but this one solves a couple of very specific usecases and gives end users a view over their backups, without compromising security for system administrators

So why is that GUI a plain local gui and not a web program ?
It’s quick and will always work regardless of html5/jquery/framework/browser evolutions.

Is there more ?
Yes, I plan to add auto upgrades and a couple of other features, feel free to open an issue at github for a feature request

Will the program stay open source ?
Yes, I pledge that this tool will stay open source as long as restic itself

Why is there no prune/forget ?
As of today I use the HTTP REST server as backend with --append-only for security reasons. Hence I did not implement them. Implementing them is a very quick work. It’s actually the GUI that would take a bit more time.

Anything more ?
Thanks to all the restic team, I can sleep well knowing my backups are secure (and restore tested ;))

10 Likes
2

NPBackup is fantastic, thank you.

I noticed I can (as far as I can tell) record the AWS secret key and AWS access key in the environment variables tab. However, they are unencrypted in the yaml file. I’m unfamiliar if the keys can be include in the URI. Is that the recommended method?

How do I compile it from start to finish with detailed instructions? I admit this question is unreasonable, feel free to link something or ignore it altogether.

Can/may I change the logo image, name, and icon if I compile it myself?

3

Hello Felix,

Thank you for your kind support.

As a matter of fact, I could (and will) add support for special environment variables that need to be encrypted. Expect this somewhere next week on the git repo.

In order to compile, you need to:

  • git clone the repository
  • install the necessary dependencies from requirements.txt file
  • install nuitka
  • change secret_keys.py
  • Run bin/compile.py script

Bear in mind that my executables for Windows are digitally signed. You’ll have to sign yours if you want to avoid antivirus programs.

As for the logo / image change, you can change anything in the customization.py file, as long as you retain the license and author info.

2 Likes
4

Thanks for the compiling info. I was able to compile it, but only with public audience. When I tried with private it would reply no private data available. I copied the PRIVATE directory to the bin directory and it compiled as private, but it still wouldn’t let me show encrypted data. I suspect it is having trouble finding the path to the PRIVATE directory, but I haven’t learned how higher level directory importing works.

I did learn I could encrypt the environment variables by adding 1 line of code to config.py and configuration.py (I suspect I only needed to change config.py). I’d do a PR, but I’m sure you know exactly what changes I made. Your project has helped me evolve from python ignorant to python novice. Thank you.

5

any plans to make it to macOS?

6

Just installed NPBackup and started to test it.

Is there a Discord or other forum to discuss NPBackup, or is the Restic forum the best place to discuss and ask questions?

7

So this week I released NPBackup v2.2 after 3 months of various testings.
Sorry for the long waiting time, but I wanted to iron out almost everything I got in various feedbacks over the months.

@Felix: Thank you for the feedback. Don’t forget to sign your executables so they pass Windows smartscreen and various AV engines.

@kapitainsky Sorry, I’ve got no macOS equipment to make tests.
Basically, everything should compile on MacOS, I just cannot guarantee it.
I’d need a KVM / Hyper-V macOS VM in order to do so, but I’m not going to just “pirate” something.
Is there perhaps an official test build or so ?

@toby9999 I’m generally quick to reply on git issues. As for the discussions, the restic forum might be a good place as long as we’re talking restic configurations that should be found in the program.
For more generic / NPBackup related questions, I enabled discussions on Github.

3 Likes
8

Thanks @deajan
– appreciate your response :slight_smile:

9

Thank you for replying. Would it be possible to provide more step by step instruction how to build this project? Information on github is rather high level for somebody who has no Python knowledge. I am not afraid of compiling stuff but would need more detailed instructions.

1 Like
10

@kapitainsky Sure, but let me quickly ask two questions first:

  • Do you understand the pro/cons of compiled vs prebuilt executables ? (I just want to make sure I’ve documented enough for people who want to start with NPBackup)
  • What platform do you aim ? (ie: I guess there’s only one MacOS ? but I’m actually not sure)
1 Like
11

Yes I do understand difference.
In addition for macOS (this is only platform I am interested in) at the moment it is only option I have so it is academic.

12

@kapitainsky I’ve written “generic” compile instructions and posted them on github.
If you get to succeed on MacOS, I’d love to have your feedback.

PS: This discussion not being restic related, please continue it in a github issue / github discussion.

2 Likes
13

Thank you and agree - see you on github

14

If anybody is interested NPBackup now can be built on macOS - there is detailed build instruction on github.

Works like a charm:) Very impressive piece of software to accompany restic.

Looks like finally we are getting restic GUI - and much more:)

1 Like
15

Silly question, I can’t find the executable for Windows

" On Windows, you can directly execute npbackup.exe or use NPBackupInstaller.exe to install NPBackup into program files and create a run schedule."

on GitHub - netinvent/npbackup: A secure and efficient file backup solution that fits both system administrators (CLI) and end users (GUI)

16

Go to release page at Releases · netinvent/npbackup · GitHub

2 Likes
17

Thanks for the quick answer!

18

Released a new version of NPBackup today, including new restic 0.16.0 internal binary.

I am currently using NPBackup in a script to backup Qemu/KVM machines too with quiesce snapshots.
If enough people are interested, I’ll rewrite to script directly into NPBackup so it becomes a full blown hypervisor backup solution.

3 Likes
19

I would love to know about your entire solution for backing up KVM Windows VMs. I use Proxmox.

1 Like
20

@smplcoder AFAIK, Promox does not use libvirt, which my solution works with.
Basically, I use a script that creates the snapshot and exports the XML file, gives NPBackup the file list to backup, than merges the snapshot.
I can share the script, but you’d need to adapt the snapshot commands to proxmox.