Restic with rclone

disconnect5852 · December 24, 2019, 5:55pm

Hi

I want to init a restic repo in my rclone remote i use command
restic -r rclone:gdrive/backup init

And i get: Fatal: create repository at rclone:gdrive/backup failed: invalid backend
If the repo is in a local directory, you need to add a local: prefix

why?

cdhowie · December 24, 2019, 6:41pm

I believe the / should be a :, like so:

restic -r rclone:gdrive:backup init

disconnect5852 · December 24, 2019, 8:47pm

Same happens with that too

jedi453 · December 25, 2019, 12:24am

Hi @disconnect5852 , welcome to the forums!

Did you configure rclone and attach google drive to it?
If you haven’t, run rclone config
If you have, it’s likely you got the rclone remote name wrong. You can see which backends are available with: rclone listremotes
The gdrive in rclone:gdrive:backup must be replaced with the name of the remote it shows. As cdhowie pointed out, you’ll need a rclone: followed by the name of the remote, followed by another : then the folder to backup to. Make sure you don’t put any spaces in between them.

Good luck,
jedi453

disconnect5852 · December 25, 2019, 1:40pm

of course. No, it’s not wrong, no space as you can see.

rawtaz · December 26, 2019, 7:13pm

If you want any relevant answers to your questions, you need to be more specific. Don’t have people pull information from you, instead provide them with relevant details.

For example; Have you configured and then been able to successfully use that backend in rclone itself (without restic)? Can you show an example where you use it successfully?

disconnect5852 · December 26, 2019, 10:51pm

i’ve provided the details, the one you asking for, i’ve answered “of course” to the same question…

I can successfully use that backend since months, for rclone copy, sync, mount, serve, etc…
Here is one example that i use,
nohup rclone serve dlna gdrive:/film --no-modtime --no-checksum --read-only --dir-cache-time 1h &> ~/rdlna.out&

rawtaz · December 27, 2019, 12:56pm

That’s not true. You had not answered the question I asked (“Have you configured and then been able to successfully use that backend in rclone itself (without restic)?”).

This example is showing usage of the gdrive backend, which is the same you’re trying to use with restic, but the /film path, which is different from the /backup path you are trying to use with restic.

Also, you have --read-only applied in this example, which makes the circumstances different from what you’re trying to do when using it through restic.

Please provide a complete example showing that you can interact with the gdrive remote and the /backup path in a non-read-only mode (i.e. where you create some files).

The reason I ask for this is to truly verify that “it works” when only the restic component is excluded from the picture

disconnect5852 · December 27, 2019, 9:43pm

without --no-modtime --no-checksum --read-only also truly works, as these added because of DLNA does not need these, to make media listing faster. As i wrote, i use it since months.

Yes, film path is different from backup, as i don’t want to backup into the folder of movies

rclone mount gdrive:/film /share/cloud/filmz --daemon --allow-other --buffer-size=128M --log-file /share/tar/mount.log
rclone mount gdrive:/moccvid /share/cloud/moccvid --daemon --allow-other --buffer-size=128M --log-file /share/tar/mount.log
nohup rclone serve dlna gdrive:/film --no-modtime --no-checksum --read-only --dir-cache-time 1h &> ~/rdlna.out&
rclone copy /share/tar/film gdrive:/film --buffer-size=256M --max-transfer 725G --log-file /share/tar/sync.log &

But i just did a simple copy to backup…
rclone copy ./ gdrive:/backup
of course it works, as it starts rclone. But restic doesn’t even invokes rclone, but writes error invalid backend BEFORE touching rclone…

also: rclone serve webdav gdrive:/film --pass pasw543 --addr lanip:8088
with rclone serve webdav gdrive:/backup --addr lanip:8088 i can upload, and download, or open files, from windows explorer…

Can anyone verify that restic EVER worked with rclone?

rawtaz · December 27, 2019, 10:16pm

That’s a silly question, there’s plenty of examples of people using restic and rclone to back up their data and having complete success.

I did look earlier today though and the examples of people using restic+rclone with Google Drive seem to be more rare though. I did find people doing it, but not many clear examples of what commands they run.

I’d love to investigate this, but I don’t have access to any Google Drive to try with

I agree, can someone else verify that they have restic+rclone working with Google Drive?

cdhowie · December 28, 2019, 4:05am

I looked at the code where this error message comes from:

github.com

restic/restic/blob/v0.9.6/internal/backend/location/location.go#L79-L99


      
          func Parse(s string) (u Location, err error) {
          	scheme := extractScheme(s)
          	u.Scheme = scheme
          
          	for _, parser := range parsers {
          		if parser.scheme != scheme {
          			continue
          		}
          
          		u.Config, err = parser.parse(s)
          		if err != nil {
          			return Location{}, err
          		}
          
          		return u, nil
          	}
          
          	// if s is not a path or contains ":", it's ambiguous
          	if !isPath(s) && strings.ContainsRune(s, ':') {
          		return Location{}, errors.New("invalid backend\nIf the repo is in a local directory, you need to add a `local:` prefix")
          	}

There is 100% no way for it to produce this error message if the repository path starts with rclone: – it will hand off to that scheme’s parser and never reach line 97.

The most likely scenario here is that you’re using a very old version of restic. Running blame on this source file, rclone support was added in v0.9.0.

So… what is the output of restic version?

Nev · December 28, 2019, 4:28am

I can confirm that at least init seems to be fine. I don’t use restic with gdrive, but do use rclone+gdrive. So for a quick test, I just made a repo directory in the gdrive root, and:

./restic_0.9.5_linux_arm -r rclone:gdrive:repo init
enter password for new repository:
enter password again:
created restic repository b41d956180 at rclone:gdrive:repo

Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is
irrecoverably lost.

cfbao · December 28, 2019, 7:33am

Absolutely. I’ve been using restic with rclone:gdrive backend every day for about a year. There’s never been any problem.

disconnect5852 · December 28, 2019, 3:59pm

0.8.3 installed from ubuntu repository, last week.

Now i’ve downloaded the newest stable binary, and works. Why there is so outdated version in repo?

cdhowie · December 28, 2019, 4:09pm

0.8.3 is in the Ubuntu LTS (Long Term Support) release. LTS only gets security and critical functionality updates. If you want newer packages, don’t use the LTS release.

disconnect5852 · December 28, 2019, 5:21pm

Now the backup works, finishes creating snapshot, except that uploads nothing to the google drive, but creates a local backup.
init also created nothing on gdrive

./restic -r rclone:gdrive/backup backup --verbose ~

jedi453 · December 28, 2019, 6:30pm

Hi disconnect5852,

You’re almost there, but you forgot to use the second : in your remote name like I discussed earlier.

Your restic init didn’t run on gdrive, instead it went through rclone to a local diectory called gdrive/backup (the first : tells restic to use an rclone remote. The second : goes straight to rclone, which needs it to know you’re not talking about a local folder, i.e. you’re using an rclone remote. That’s why the two : are both needed. )

You’ll have to start over with google drive, or move your current backup to google drive.
To start from scratch, delete the gdrive folder in your home directory, then run this :

./restic init -r rclone:gdrive:/backup
./restic backup - r rclone:gdrive:/backup --verbose ~

If you want to upload the existing one, do this INSTEAD:

rclone copy gdrive/backup gdrive:
./restic snapshots -r rclone:grive:backup

Good luck,
Jedi453

rawtaz · December 28, 2019, 7:55pm

Before you question and criticize as much as you seem to like to do, please take a moment and consider why things might be the way they are. In this specific case the answer is obvious; The restic project is not what maintains the Ubuntu binaries, not to mention what @cdhowie wrote, the fact that Ubuntu LTS will never have very up to date versions of software in its packages.

Also, while I’m at it, if you had taken the time to tell us the version of restic that you are running, we could have told you much sooner that this was the reason for your problems. Again; Try to proactively provide information when you are asking for help, it makes it easier for everyone.