Which protocol should I choose for remote Linux backups?

Getting Help
1

Hi,
I successfully used restic to backup my personal data to onedrive via rclone from my Synology NAS (quite cool by the way, Hyperbackup does not offer that target :slightly_smiling_face:).

So I thought about using restic also to backup VPS data. I am wondering about which target option would work best for that use case, I currently prototyped backups via sftp and pruning against the local repo from the target server which seem to work so far and even without rclone.

What does the community recommend as preferred target option? Are all target options considered equally good? I guess sftp is not that fast, but works without running additional software on the target host and securely over untrusted connections.

If using another VPS as target, I could also run the restic server, it’s behind a firewall so I guess http would be OK as backup content is encrypted anyhow. In case of http, could I leave out the auth credentials and run in append only mode and forget and prune against the local repo from the target VPS?
The restic server approach sounds a bit like using rsync for encrypted data via the unencrypted rsync daemon vs rsync over ssh. In case of rsync ssh encryption slows down performance in comparison quite a bit, will that be similar with restic over sftp compared to restic server?
If I turn on TLS on the restic server, will that be comparable speed-wise to sftp?

Or is S3 via minio the preferred approach?

I also have access to an Artifactory deployment, it might work via rclone’s webdav integration, but not sure how reliable and performant that would be in comparison (but I would not be responsible for hosting that backend :sweat_smile:).

Overall, will using rclone targets be slower as they require another binary in the mix? I guess it is not that different than shelling out to sftp.

Thanks in advance
Konrad

2

Lot’s of questions! I’ll answer some of them:

There is no such thing as “restic server” - it’s called REST server, see GitHub - restic/rest-server: Rest Server is a high performance HTTP server that implements restic's REST backend API. to run it.

Yes, you could (although I fail to see why you’d want to do that instead of running it over HTTPS), but please make sure to note the security considerations here (since the control traffic won’t be encrypted if you run over HTTP alone): Removing backup snapshots — restic 0.16.3 documentation

The answer to your question of what to run simply depends on where you want to store your backups. Where do you want to store them? That’s the main driver. If on another VPS, then I’d recommend REST server, otherwise if on somewhere you can run a container, I’d still recommend REST server, but it’s also possible to send stuff directly to most cloud storages. So it depends on your needs, please elaborate.

1 Like
3

Thanks for your fast and detailed response.
Yes, I had restic/rest-server in mind, thanks for correcting that.

My main concern is a robust backup process of data and low maintenance overhead for hosting the solution.

I don’t have an existing S3 account I can just use, so I thougt Artifactory would give me the closest user experience in comparison as I don’t need to do capacity planning on my end nor need to host it.
But I did not find a tutorial/documentation for Artifactory or anyone mention using it.
Has anyone used Artifactory as storage backend yet and can share experiences? I might need to test, if it can be used via it’s webdav API in more detail. Initially, I had tried it once and noticed some wired behavior that I could only restore the latest snapshot, if I used an ID instead of latest, there was an error message that this ref was found multiple times and the restore failed. (sry cannot recall the exact message from my head any more, might need to check details again later)
I might have done a user error back than, afterwards I switched to sftp and saw no issues any more.

But I am also not dealing with terabytes of data, mostly database dumps and config files so just provision a decently sized VPS would not be that big of a problem. If that is the more reliable approach, I could go down that route and maybe setup rsync to a second VPS in a different region to have a backup if the primary backup server goes down.

Regarding the additional overhead of running the rest server, it looks quite straight forward from the docs, also not to many releases to keep up with.

But from your answer I get that you would prefer the rest server over sftp, I think I try that out as well, thanks :slightly_smiling_face:

4

Well you always have the rclone backend in restic which allows you to use any storage that rclone supports. Other than that, go for what is suitable for you. But yes, I do like REST server very much, it works flawlessly. I did use SFTP before, but haven’t looked back.

4 Likes
5

FYI, I reproduced the problem with Artifactory and reported it here, I am not sure if restic, Artifactory or rclone is the problem here, I am sorry if this is not the right location, but I don’t know how to debug it further on my own:

$ DEBUG_LOG=restic-prune-debug.log restic forget --prune 18977eee -r rclone:artifactory:/restic
debug log file restic-prune-debug.log
debug enabled
enter password for repository: 
repository 23ae2878 opened (repository version 2) successfully, password is correct
Ignoring "18977eee": multiple IDs with prefix "18977eee" found

$ DEBUG_LOG=restic-restore-debug.log restic restore 18977eee -r rclone:artifactory:/restic --target /tmp/restore/
debug log file restic-restore-debug.log
debug enabled
enter password for repository: 
repository 23ae2878 opened (repository version 2) successfully, password is correct
invalid id "18977eee": multiple IDs with prefix "18977eee" found

Restoring “latest” or using the fuse mount worked for some reason:

$ DEBUG_LOG=restic-restore-debug.log restic restore latest -r rclone:artifactory:/restic --target /tmp/restore/
debug log file restic-restore-debug.log
debug enabled
enter password for repository: 
repository 23ae2878 opened (repository version 2) successfully, password is correct
restoring <Snapshot 18977eee of [/tmp/restic-init-debug.log] at 2022-09-19 08:26:35.880118336 +0200 CEST by xxx@xxx> to /tmp/restore/
6

That error message has nothing to do with the storage provider. It simply says that there are two or more snapshots whose id starts with 18977eee so restic doesn’t know which one you’re referring to. Some commands might just pick the one of these snapshot but the selection is essentially random.

You have to use the full snapshot id, which is 64 characters long.

7

Thanks Michael,

I was thinking the same initially, but the test repo only contained a single snapshot and using a long id did not help either, I get the same error.
I tried the same via sftp and never had such behavior before.
If I only run a backup once, I assumed that there is no possibility of a conflicting snapshot id or do I miss something?

Thanks
Konrad

8

I also use rest-server between my machines at home, it’s definitely the fastest, and it doesn’t require much configuration: it’s running inside a docker container.

Between my VPS I also use rest-server but with wireguard in between to secure the transmission (easier than setting up a CA for https)

1 Like
9

That’s an interesting approach, thanks. I used to have tailscale on some of my servers, so at least in those environments it should work fine, if I directly connect peer to peer and not use exit nodes and expose subnets.
I also used rest server with tls, but always needed to attach --insecure-tls which is slightly inconvenient for cli usage. Anyway, I guess having TLS is the more secure approach as man in the middle attacks are less likely, e.g. if you have it running in the LAN. Considering the number of devices in a local networks today, I tend to no longer consider everything safe which is running on my LAN, too many black bloxes like washing machine, dish washer and so on which I don’t know, if they are patched against latest vulnerabilities.

10

Can’t you just have it use Let’s Encrypt certificates instead, so that you know it’s a proper certificate and don’t need to use the --insecure-tls option?

11

I think let’s encrypt will only work for systems exposed to the internet and with a DNS record attached. In the mentioned setup I can use Artifactory as hosted option or run my own server but behind a firewall, so to my knowledge let’s encrypt would not work, otherwise you are right, using a real cert should resolve that problem.

There is no env var I can just put into ~/.bashrc to disable tls validation globally by any chance? :sweat_smile:

12

Yeah, but how is that a problem? Don’t you have a domain.com that you can add a backup.domain.com FQDN to that you can then set up Let’s Encrypt for and let it be accessible from the Internet?

You don’t have to let the REST-server be public if you don’t want to, and you can just have the ingress access be open for the renewal of certificates and nothing else if you want to. There’s many possibilities to set this up without having to use --insecure-tls :+1:

Put alias restic='restic --insecure-tls' in your ~/.profile or the ~/.bashrc you mentioned, then you’ll have a restic command that automatically includes this option. Another option is to simply write a script that wraps the restic command for you.

13

Seriously, using --insecure-tls is probably as safe as http. An attacker can replace the backup host with DNS poisoning and the client pushing backups to it will see nothing at all.

I used to manage my own CA, it’s not that bad. I recently discovered smallstep certificate manager that simply runs in a docker container and it’s simpler to manage

14

Thanks,
yeah, I think you’re right. If I stick with self signed certs, I probably want to define the --cacert flag and use the alias to add it globally. I thought tls would at least prevent others from sniffing my basic auth headers, but if they can fake the backend they could also just get a valid request and decode the headers.

Regarding let’s encrypt, I did not consider that, thanks. I manage an additional set Iof servers in a different project where I don’t even have control over DNS records at all, but in that case I can fallback to sftp or the cacert flag.

Regarding artifactory, I think it’s a problem with their webdav implementation, I tested against a webdav server running on my Synology NAS and that worked without those strange ID messages. Probably I just not use Artifactory for my scenarios, running one or two additional VMs as backup storage might be the more reliable approach.

Thanks so much guys, I learned a lot about the different restic setups the last week :slightly_smiling_face:

15

Even if you don’t have a single domain yourself, you can work around that by simply using one of the free DynDNS domains out there. I’ve had great success with http://dynu.com - just sign up there and set up a dynamic domain name, that you can then either just set an IP for it in their UI once, or run a simple DynDNS agent on your system to have it always keep this FQDN up to date with whatever IP is current at the time.

1 Like
16

What does restic list snapshots return. Does the debug log contain any unable to parse %v as an ID warnings?

17

Yes it does show that.

The section is

2022/09/24 20:57:39 rest/rest.go:481    rest.(*Backend).listv2      38      parsing API v2 response
2022/09/24 20:57:39 repository/repository.go:779        repository.(*Repository).List.func1 38      unable to parse 72e6921f9ba1ec878468cb71ed8447ae9dd29e2b4d3ef9d7f2eb7c05703bb8d2.sha256 as an ID
2022/09/24 20:57:39 restic/lock.go:324  restic.ForAllLocks.func2    43      load lock 72e6921f9ba1ec878468cb71ed8447ae9dd29e2b4d3ef9d7f2eb7c05703bb8d2

Here are the full steps as background information.

$ restic snapshots -r rclone:artifactory:restic
enter password for repository:
repository 23ae2878 opened (repository version 2) successfully, password is correct
created new cache in /home/xxx/.cache/restic
ID        Time                 Host        Tags        Paths---------------------------------------------------------------------------------
18977eee  2022-09-19 08:26:35  xxx                    /tmp/restic-init-debug.log
---------------------------------------------------------------------------------
1 snapshots

$ restic list snapshots -r rclone:artifactory:restic
enter password for repository:
repository 23ae2878 opened (repository version 2) successfully, password is correct
18977eeeeed126528312aabc0ae0293af91fa04fb53a7d79db3c2e4b2e09893f

$ restic restore 18977eee --target restore -r rclone:artifactory:restic
enter password for repository:
repository 23ae2878 opened (repository version 2) successfully, password is correct
invalid id "18977eee": multiple IDs with prefix "18977eee" found

$ restic restore 18977eeeeed126528312aabc0ae0293af91fa04fb53a7d79db3c2e4b2e09893f --target restore -r rclone:artifactory:restic
enter password for repository:
repository 23ae2878 opened (repository version 2) successfully, password is correct
invalid id "18977eeeeed126528312aabc0ae0293af91fa04fb53a7d79db3c2e4b2e09893f": multiple IDs with prefix "18977eeeeed126528312aabc0ae0293af91fa04fb53a7d79db3c2e4b2e09893f" found







$ DEBUG_LOG=debug.log restic restore 18977eeeeed126528312aabc0ae0293af91fa04fb53a7d79db3c2e4b2e09893f --target restore -r rclone:artifactory:restic
debug log file debug.log
debug enabled
enter password for repository:
repository 23ae2878 opened (repository version 2) successfully, password is correct
invalid id "18977eeeeed126528312aabc0ae0293af91fa04fb53a7d79db3c2e4b2e09893f": multiple IDs with prefix "18977eeeeed126528312aabc0ae0293af91fa04fb53a7d79db3c2e4b2e09893f" found






$ cat debug.log
2022/09/24 20:57:31 restic/main.go:95   main.main       1  main []string{"restic", "restore", "18977eeeeed126528312aabc0ae0293af91fa04fb53a7d79db3c2e4b2e09893f", "--target", "restore", "-r", "rclone:artifactory:restic"}
2022/09/24 20:57:31 restic/main.go:96   main.main       1  restic 0.14.0 compiled with go1.19 on linux/amd64
2022/09/24 20:57:31 restic/cmd_restore.go:120   main.runRestore     1       restore 18977eeeeed126528312aabc0ae0293af91fa04fb53a7d79db3c2e4b2e09893f to restore
2022/09/24 20:57:31 restic/global.go:690        main.open  parsing location rclone:artifactory:restic
2022/09/24 20:57:31 restic/global.go:681        main.parseConfig    1       opening rest repository at rclone.Config{Program:"rclone", Args:"serve restic --stdio --b2-hard-delete", Remote:"artifactory:restic", Connections:0x5, Timeout:60000000000}
2022/09/24 20:57:31 rclone/backend.go:160       rclone.newBackend   1       running command: rclone [serve restic --stdio --b2-hard-delete artifactory:restic]
2022/09/24 20:57:31 rclone/backend.go:198       rclone.newBackend.func2     10      waiting for error result
2022/09/24 20:57:31 rclone/backend.go:213       rclone.newBackend.func3     11      monitoring command to cancel first HTTP request context
2022/09/24 20:57:31 debug/round_tripper.go:94   debug.loggingRoundTripper.RoundTrip 1       ------------  HTTP REQUEST -----------
GET /file-5577006791947779410 HTTP/1.1
Host: localhost
User-Agent: Go-http-client/1.1
Accept: application/vnd.x.restic.rest.v2
Accept-Encoding: gzip


2022/09/24 20:57:31 rclone/backend.go:175       rclone.newBackend.func1     65      new connection requested, tcp localhost:80
2022/09/24 20:57:31 debug/round_tripper.go:111  debug.loggingRoundTripper.RoundTrip 1       ------------  HTTP RESPONSE ----------
HTTP/2.0 404 Not Found
Content-Length: 10
Accept-Ranges: bytes
Content-Type: text/plain; charset=utf-8
Date: Sat, 24 Sep 2022 18:57:31 GMT
Server: rclone/v1.59.2
X-Content-Type-Options: nosniff


2022/09/24 20:57:31 rclone/backend.go:247       rclone.newBackend   1       HTTP status "404 Not Found" returned, moving instance to background
2022/09/24 20:57:31 rclone/backend.go:216       rclone.newBackend.func3     11      context has been cancelled, returning
2022/09/24 20:57:31 debug/round_tripper.go:94   debug.loggingRoundTripper.RoundTrip 1       ------------  HTTP REQUEST -----------
HEAD /config HTTP/1.1
Host: localhost
User-Agent: Go-http-client/1.1
Accept: application/vnd.x.restic.rest.v2


2022/09/24 20:57:31 debug/round_tripper.go:111  debug.loggingRoundTripper.RoundTrip 1       ------------  HTTP RESPONSE ----------
HTTP/2.0 200 OK
Content-Length: 155
Accept-Ranges: bytes
Date: Sat, 24 Sep 2022 18:57:31 GMT
Server: rclone/v1.59.2


2022/09/24 20:57:31 repository/index.go:404     repository.(*Index).Finalize        1       finalizing index
2022/09/24 20:57:35 debug/round_tripper.go:94   debug.loggingRoundTripper.RoundTrip 1       ------------  HTTP REQUEST -----------
GET /keys/ HTTP/1.1
Host: localhost
User-Agent: Go-http-client/1.1
Accept: application/vnd.x.restic.rest.v2
Accept-Encoding: gzip


2022/09/24 20:57:35 debug/round_tripper.go:111  debug.loggingRoundTripper.RoundTrip 1       ------------  HTTP RESPONSE ----------
HTTP/2.0 200 OK
Content-Length: 182
Accept-Ranges: bytes
Content-Type: application/vnd.x.restic.rest.v2
Date: Sat, 24 Sep 2022 18:57:35 GMT
Server: rclone/v1.59.2


2022/09/24 20:57:35 rest/rest.go:481    rest.(*Backend).listv2      1       parsing API v2 response
2022/09/24 20:57:35 repository/key.go:151       repository.SearchKey.func1  1       trying key "403595b9f36a976e9a01e0b4ed5732f05c33099a5d660ccaf75509f58c5833c2"
2022/09/24 20:57:35 rest/rest.go:259    rest.(*Backend).openReader  1       Load <key/403595b9f3>, length 0, offset 0
2022/09/24 20:57:35 rest/rest.go:283    rest.(*Backend).openReader  1       Load(<key/403595b9f3>) send range bytes=0-
2022/09/24 20:57:35 debug/round_tripper.go:94   debug.loggingRoundTripper.RoundTrip 1       ------------  HTTP REQUEST -----------
GET /keys/403595b9f36a976e9a01e0b4ed5732f05c33099a5d660ccaf75509f58c5833c2 HTTP/1.1
Host: localhost
User-Agent: Go-http-client/1.1
Accept: application/vnd.x.restic.rest.v2
Range: bytes=0-


2022/09/24 20:57:35 debug/round_tripper.go:111  debug.loggingRoundTripper.RoundTrip 1       ------------  HTTP RESPONSE ----------
HTTP/2.0 206 Partial Content
Content-Length: 447
Accept-Ranges: bytes
Content-Range: bytes 0-446/447
Content-Type: text/plain; charset=utf-8
Date: Sat, 24 Sep 2022 18:57:35 GMT
Server: rclone/v1.59.2


2022/09/24 20:57:36 repository/key.go:164       repository.SearchKey.func1  1       successfully opened key 403595b9f36a976e9a01e0b4ed5732f05c33099a5d660ccaf75509f58c5833c2
2022/09/24 20:57:36 repository/repository.go:180        repository.(*Repository).LoadUnpacked       1       load config with id 0000000000000000000000000000000000000000000000000000000000000000
2022/09/24 20:57:36 rest/rest.go:259    rest.(*Backend).openReader  1       Load <config/0000000000>, length 0, offset 02022/09/24 20:57:36 rest/rest.go:283    rest.(*Backend).openReader  1       Load(<config/0000000000>) send range bytes=0-
2022/09/24 20:57:36 debug/round_tripper.go:94   debug.loggingRoundTripper.RoundTrip 1       ------------  HTTP REQUEST -----------
GET /config HTTP/1.1
Host: localhost
User-Agent: Go-http-client/1.1
Accept: application/vnd.x.restic.rest.v2
Range: bytes=0-


2022/09/24 20:57:36 debug/round_tripper.go:111  debug.loggingRoundTripper.RoundTrip 1       ------------  HTTP RESPONSE ----------
HTTP/2.0 206 Partial Content
Content-Length: 155
Accept-Ranges: bytes
Content-Range: bytes 0-154/155
Content-Type: application/octet-stream
Date: Sat, 24 Sep 2022 18:57:36 GMT
Server: rclone/v1.59.2


2022/09/24 20:57:36 debug/round_tripper.go:94   debug.loggingRoundTripper.RoundTrip 1       ------------  HTTP REQUEST -----------
GET /keys/ HTTP/1.1
Host: localhost
User-Agent: Go-http-client/1.1
Accept: application/vnd.x.restic.rest.v2
Accept-Encoding: gzip


2022/09/24 20:57:36 debug/round_tripper.go:111  debug.loggingRoundTripper.RoundTrip 1       ------------  HTTP RESPONSE ----------
HTTP/2.0 200 OK
Content-Length: 182
Accept-Ranges: bytes
Content-Type: application/vnd.x.restic.rest.v2
Date: Sat, 24 Sep 2022 18:57:36 GMT
Server: rclone/v1.59.2


2022/09/24 20:57:36 rest/rest.go:481    rest.(*Backend).listv2      1       parsing API v2 response
2022/09/24 20:57:36 repository/key.go:151       repository.SearchKey.func1  1       trying key "403595b9f36a976e9a01e0b4ed5732f05c33099a5d660ccaf75509f58c5833c2"
2022/09/24 20:57:36 rest/rest.go:259    rest.(*Backend).openReader  1       Load <key/403595b9f3>, length 0, offset 0
2022/09/24 20:57:36 rest/rest.go:283    rest.(*Backend).openReader  1       Load(<key/403595b9f3>) send range bytes=0-
2022/09/24 20:57:36 debug/round_tripper.go:94   debug.loggingRoundTripper.RoundTrip 1       ------------  HTTP REQUEST -----------
GET /keys/403595b9f36a976e9a01e0b4ed5732f05c33099a5d660ccaf75509f58c5833c2 HTTP/1.1
Host: localhost
User-Agent: Go-http-client/1.1
Accept: application/vnd.x.restic.rest.v2
Range: bytes=0-


2022/09/24 20:57:36 debug/round_tripper.go:111  debug.loggingRoundTripper.RoundTrip 1       ------------  HTTP RESPONSE ----------
HTTP/2.0 206 Partial Content
Content-Length: 447
Accept-Ranges: bytes
Content-Range: bytes 0-446/447
Content-Type: text/plain; charset=utf-8
Date: Sat, 24 Sep 2022 18:57:36 GMT
Server: rclone/v1.59.2


2022/09/24 20:57:37 repository/key.go:164       repository.SearchKey.func1  1       successfully opened key 403595b9f36a976e9a01e0b4ed5732f05c33099a5d660ccaf75509f58c5833c2
2022/09/24 20:57:37 repository/repository.go:180        repository.(*Repository).LoadUnpacked       1       load config with id 0000000000000000000000000000000000000000000000000000000000000000
2022/09/24 20:57:37 rest/rest.go:259    rest.(*Backend).openReader  1       Load <config/0000000000>, length 0, offset 02022/09/24 20:57:37 rest/rest.go:283    rest.(*Backend).openReader  1       Load(<config/0000000000>) send range bytes=0-
2022/09/24 20:57:37 debug/round_tripper.go:94   debug.loggingRoundTripper.RoundTrip 1       ------------  HTTP REQUEST -----------
GET /config HTTP/1.1
Host: localhost
User-Agent: Go-http-client/1.1
Accept: application/vnd.x.restic.rest.v2
Range: bytes=0-


2022/09/24 20:57:38 debug/round_tripper.go:111  debug.loggingRoundTripper.RoundTrip 1       ------------  HTTP RESPONSE ----------
HTTP/2.0 206 Partial Content
Content-Length: 155
Accept-Ranges: bytes
Content-Range: bytes 0-154/155
Content-Type: application/octet-stream
Date: Sat, 24 Sep 2022 18:57:38 GMT
Server: rclone/v1.59.2


2022/09/24 20:57:38 debug/round_tripper.go:94   debug.loggingRoundTripper.RoundTrip 1       ------------  HTTP REQUEST -----------
GET /keys/ HTTP/1.1
Host: localhost
User-Agent: Go-http-client/1.1
Accept: application/vnd.x.restic.rest.v2
Accept-Encoding: gzip


2022/09/24 20:57:38 debug/round_tripper.go:111  debug.loggingRoundTripper.RoundTrip 1       ------------  HTTP RESPONSE ----------
HTTP/2.0 200 OK
Content-Length: 182
Accept-Ranges: bytes
Content-Type: application/vnd.x.restic.rest.v2
Date: Sat, 24 Sep 2022 18:57:38 GMT
Server: rclone/v1.59.2


2022/09/24 20:57:38 rest/rest.go:481    rest.(*Backend).listv2      1       parsing API v2 response
2022/09/24 20:57:38 repository/key.go:151       repository.SearchKey.func1  1       trying key "403595b9f36a976e9a01e0b4ed5732f05c33099a5d660ccaf75509f58c5833c2"
2022/09/24 20:57:38 rest/rest.go:259    rest.(*Backend).openReader  1       Load <key/403595b9f3>, length 0, offset 0
2022/09/24 20:57:38 rest/rest.go:283    rest.(*Backend).openReader  1       Load(<key/403595b9f3>) send range bytes=0-
2022/09/24 20:57:38 debug/round_tripper.go:94   debug.loggingRoundTripper.RoundTrip 1       ------------  HTTP REQUEST -----------
GET /keys/403595b9f36a976e9a01e0b4ed5732f05c33099a5d660ccaf75509f58c5833c2 HTTP/1.1
Host: localhost
User-Agent: Go-http-client/1.1
Accept: application/vnd.x.restic.rest.v2
Range: bytes=0-


2022/09/24 20:57:38 debug/round_tripper.go:111  debug.loggingRoundTripper.RoundTrip 1       ------------  HTTP RESPONSE ----------
HTTP/2.0 206 Partial Content
Content-Length: 447
Accept-Ranges: bytes
Content-Range: bytes 0-446/447
Content-Type: text/plain; charset=utf-8
Date: Sat, 24 Sep 2022 18:57:38 GMT
Server: rclone/v1.59.2


2022/09/24 20:57:39 repository/key.go:164       repository.SearchKey.func1  1       successfully opened key 403595b9f36a976e9a01e0b4ed5732f05c33099a5d660ccaf75509f58c5833c2
2022/09/24 20:57:39 repository/repository.go:180        repository.(*Repository).LoadUnpacked       1       load config with id 0000000000000000000000000000000000000000000000000000000000000000
2022/09/24 20:57:39 rest/rest.go:259    rest.(*Backend).openReader  1       Load <config/0000000000>, length 0, offset 02022/09/24 20:57:39 rest/rest.go:283    rest.(*Backend).openReader  1       Load(<config/0000000000>) send range bytes=0-
2022/09/24 20:57:39 debug/round_tripper.go:94   debug.loggingRoundTripper.RoundTrip 1       ------------  HTTP REQUEST -----------
GET /config HTTP/1.1
Host: localhost
User-Agent: Go-http-client/1.1
Accept: application/vnd.x.restic.rest.v2
Range: bytes=0-


2022/09/24 20:57:39 debug/round_tripper.go:111  debug.loggingRoundTripper.RoundTrip 1       ------------  HTTP RESPONSE ----------
HTTP/2.0 206 Partial Content
Content-Length: 155
Accept-Ranges: bytes
Content-Range: bytes 0-154/155
Content-Type: application/octet-stream
Date: Sat, 24 Sep 2022 18:57:39 GMT
Server: rclone/v1.59.2


2022/09/24 20:57:39 cache/cache.go:104  cache.New       1  using cache dir /home/xxx/.cache/restic/23ae2878b376cd05e3463f6e3f2b4f71104b4c64df0ccdf00ee686fefb42b4a82022/09/24 20:57:39 repository/repository.go:156        repository.(*Repository).UseCache   1       using cache
2022/09/24 20:57:39 cache/cache.go:226  cache.OlderThan 1  0 old cache dirs found
2022/09/24 20:57:39 debug/round_tripper.go:94   debug.loggingRoundTripper.RoundTrip 38      ------------  HTTP REQUEST -----------
GET /locks/ HTTP/1.1
Host: localhost
User-Agent: Go-http-client/1.1
Accept: application/vnd.x.restic.rest.v2
Accept-Encoding: gzip


2022/09/24 20:57:39 debug/round_tripper.go:111  debug.loggingRoundTripper.RoundTrip 38      ------------  HTTP RESPONSE ----------
HTTP/2.0 200 OK
Content-Length: 182
Accept-Ranges: bytes
Content-Type: application/vnd.x.restic.rest.v2
Date: Sat, 24 Sep 2022 18:57:39 GMT
Server: rclone/v1.59.2


2022/09/24 20:57:39 rest/rest.go:481    rest.(*Backend).listv2      38      parsing API v2 response
2022/09/24 20:57:39 repository/repository.go:779        repository.(*Repository).List.func1 38      unable to parse 72e6921f9ba1ec878468cb71ed8447ae9dd29e2b4d3ef9d7f2eb7c05703bb8d2.sha256 as an ID
2022/09/24 20:57:39 restic/lock.go:324  restic.ForAllLocks.func2    43      load lock 72e6921f9ba1ec878468cb71ed8447ae9dd29e2b4d3ef9d7f2eb7c05703bb8d2
2022/09/24 20:57:39 repository/repository.go:180        repository.(*Repository).LoadUnpacked       43      load lock with id 72e6921f9ba1ec878468cb71ed8447ae9dd29e2b4d3ef9d7f2eb7c05703bb8d2
2022/09/24 20:57:39 cache/backend.go:179        cache.(*Backend).Load       43      Load(<lock/72e6921f9b>, 0, 0): delegating to backend
2022/09/24 20:57:39 rest/rest.go:259    rest.(*Backend).openReader  43      Load <lock/72e6921f9b>, length 0, offset 0
2022/09/24 20:57:39 rest/rest.go:283    rest.(*Backend).openReader  43      Load(<lock/72e6921f9b>) send range bytes=0-
2022/09/24 20:57:39 debug/round_tripper.go:94   debug.loggingRoundTripper.RoundTrip 43      ------------  HTTP REQUEST -----------
GET /locks/72e6921f9ba1ec878468cb71ed8447ae9dd29e2b4d3ef9d7f2eb7c05703bb8d2 HTTP/1.1
Host: localhost
User-Agent: Go-http-client/1.1
Accept: application/vnd.x.restic.rest.v2
Range: bytes=0-


2022/09/24 20:57:39 debug/round_tripper.go:111  debug.loggingRoundTripper.RoundTrip 43      ------------  HTTP RESPONSE ----------
HTTP/2.0 206 Partial Content
Content-Length: 155
Accept-Ranges: bytes
Content-Range: bytes 0-154/155
Content-Type: application/octet-stream
Date: Sat, 24 Sep 2022 18:57:39 GMT
Server: rclone/v1.59.2


2022/09/24 20:57:39 restic/json.go:25   restic.SaveJSONUnpacked     1       save new blob lock
2022/09/24 20:57:39 debug/round_tripper.go:94   debug.loggingRoundTripper.RoundTrip 1       ------------  HTTP REQUEST -----------
POST /locks/7e1dec13333523fd128e9856084cad318fa0769b2baeb7b3e845ca0842812617 HTTP/1.1
Host: localhost
User-Agent: Go-http-client/1.1
Content-Length: 158
Accept: application/vnd.x.restic.rest.v2
Content-Type: application/octet-stream
Accept-Encoding: gzip


2022/09/24 20:57:39 debug/round_tripper.go:111  debug.loggingRoundTripper.RoundTrip 1       ------------  HTTP RESPONSE ----------
HTTP/2.0 200 OK
Content-Length: 0
Accept-Ranges: bytes
Date: Sat, 24 Sep 2022 18:57:39 GMT
Server: rclone/v1.59.2


2022/09/24 20:57:39 repository/repository.go:494        repository.(*Repository).SaveUnpacked       1       blob <lock/7e1dec1333> saved
2022/09/24 20:57:39 debug/round_tripper.go:94   debug.loggingRoundTripper.RoundTrip 103     ------------  HTTP REQUEST -----------
GET /locks/ HTTP/1.1
Host: localhost
User-Agent: Go-http-client/1.1
Accept: application/vnd.x.restic.rest.v2
Accept-Encoding: gzip


2022/09/24 20:57:39 debug/round_tripper.go:111  debug.loggingRoundTripper.RoundTrip 103     ------------  HTTP RESPONSE ----------
HTTP/2.0 200 OK
Content-Length: 362
Accept-Ranges: bytes
Content-Type: application/vnd.x.restic.rest.v2
Date: Sat, 24 Sep 2022 18:57:39 GMT
Server: rclone/v1.59.2


2022/09/24 20:57:39 rest/rest.go:481    rest.(*Backend).listv2      103     parsing API v2 response
2022/09/24 20:57:39 repository/repository.go:779        repository.(*Repository).List.func1 103     unable to parse 72e6921f9ba1ec878468cb71ed8447ae9dd29e2b4d3ef9d7f2eb7c05703bb8d2.sha256 as an ID
2022/09/24 20:57:39 repository/repository.go:779        repository.(*Repository).List.func1 103     unable to parse 7e1dec13333523fd128e9856084cad318fa0769b2baeb7b3e845ca0842812617.sha256 as an ID
2022/09/24 20:57:39 restic/lock.go:324  restic.ForAllLocks.func2    105     load lock 72e6921f9ba1ec878468cb71ed8447ae9dd29e2b4d3ef9d7f2eb7c05703bb8d2
2022/09/24 20:57:39 repository/repository.go:180        repository.(*Repository).LoadUnpacked       105     load lock with id 72e6921f9ba1ec878468cb71ed8447ae9dd29e2b4d3ef9d7f2eb7c05703bb8d2
2022/09/24 20:57:39 cache/backend.go:179        cache.(*Backend).Load       105     Load(<lock/72e6921f9b>, 0, 0): delegating to backend
2022/09/24 20:57:39 rest/rest.go:259    rest.(*Backend).openReader  105     Load <lock/72e6921f9b>, length 0, offset 0
2022/09/24 20:57:39 rest/rest.go:283    rest.(*Backend).openReader  105     Load(<lock/72e6921f9b>) send range bytes=0-
2022/09/24 20:57:39 debug/round_tripper.go:94   debug.loggingRoundTripper.RoundTrip 105     ------------  HTTP REQUEST -----------
GET /locks/72e6921f9ba1ec878468cb71ed8447ae9dd29e2b4d3ef9d7f2eb7c05703bb8d2 HTTP/1.1
Host: localhost
User-Agent: Go-http-client/1.1
Accept: application/vnd.x.restic.rest.v2
Range: bytes=0-


2022/09/24 20:57:39 debug/round_tripper.go:111  debug.loggingRoundTripper.RoundTrip 105     ------------  HTTP RESPONSE ----------
HTTP/2.0 206 Partial Content
Content-Length: 155
Accept-Ranges: bytes
Content-Range: bytes 0-154/155
Content-Type: application/octet-stream
Date: Sat, 24 Sep 2022 18:57:39 GMT
Server: rclone/v1.59.2


2022/09/24 20:57:39 restic/lock.go:46   main.lockRepositorycreate lock 0xc000549810 (exclusive false)
2022/09/24 20:57:39 restic/lock.go:50   main.lockRepositorystart goroutine for lock refresh
2022/09/24 20:57:39 restic/lock.go:66   main.refreshLocks  130      start
2022/09/24 20:57:39 debug/round_tripper.go:94   debug.loggingRoundTripper.RoundTrip 1       ------------  HTTP REQUEST -----------
GET /snapshots/ HTTP/1.1
Host: localhost
User-Agent: Go-http-client/1.1
Accept: application/vnd.x.restic.rest.v2
Accept-Encoding: gzip


2022/09/24 20:57:39 debug/round_tripper.go:111  debug.loggingRoundTripper.RoundTrip 1       ------------  HTTP RESPONSE ----------
HTTP/2.0 200 OK
Content-Length: 182
Accept-Ranges: bytes
Content-Type: application/vnd.x.restic.rest.v2
Date: Sat, 24 Sep 2022 18:57:39 GMT
Server: rclone/v1.59.2


2022/09/24 20:57:39 rest/rest.go:481    rest.(*Backend).listv2      1       parsing API v2 response
2022/09/24 20:57:39 restic/lock.go:126  main.unlockAll  1  unlocking 1 locks
2022/09/24 20:57:39 cache/backend.go:37 cache.(*Backend).Remove     1       cache Remove(<lock/7e1dec1333>)
2022/09/24 20:57:39 debug/round_tripper.go:94   debug.loggingRoundTripper.RoundTrip 1       ------------  HTTP REQUEST -----------
DELETE /locks/7e1dec13333523fd128e9856084cad318fa0769b2baeb7b3e845ca0842812617 HTTP/1.1
Host: localhost
User-Agent: Go-http-client/1.1
Accept: application/vnd.x.restic.rest.v2
Accept-Encoding: gzip


2022/09/24 20:57:40 debug/round_tripper.go:111  debug.loggingRoundTripper.RoundTrip 1       ------------  HTTP RESPONSE ----------
HTTP/2.0 200 OK
Accept-Ranges: bytes
Date: Sat, 24 Sep 2022 18:57:40 GMT
Server: rclone/v1.59.2
Content-Length: 0


2022/09/24 20:57:40 restic/lock.go:132  main.unlockAll  1  successfully removed lock
18

A wild guess is that Artifactory calculates a SHA256 sum for the snapshot files and creates corresponding files or some form of representation them with .sha256 suffix, so in the case of 18977eeeeed126528312aabc0ae0293af91fa04fb53a7d79db3c2e4b2e09893f there is both a file named 18977eeeeed126528312aabc0ae0293af91fa04fb53a7d79db3c2e4b2e09893f and a file named 18977eeeeed126528312aabc0ae0293af91fa04fb53a7d79db3c2e4b2e09893f.sha256 in the snapshots/ folder?

Is there any way you can list the contents of the snapshots/ folder in the repository, I mean outside of restic?

1 Like
19

Oh wow, thanks, you are right, Artifactory seem to just generate that checksum for every uploaded file automatically (https://www.jfrog.com/confluence/display/JFROG/SHA-256+Support)……). I wasn’t aware of that, but if I compare my snapshots of a backup stored on onedrive vs artifactory it becomes visible…

$ rclone ls artifactory:/restic/snapshots
213 18977eeeeed126528312aabc0ae0293af91fa04fb53a7d79db3c2e4b2e09893f
64 18977eeeeed126528312aabc0ae0293af91fa04fb53a7d79db3c2e4b2e09893f.sha256

vs.:

$ rclone ls onedrive:/backup/synology/snapshots
291 1287f0b50c414c1db1ecf064a2fd23a34a75b8d90f7a53cb03f48012fcff9c59
291 16dbee7572420cf5e2e473959ac0c80d4b0132101f12caaa8e11728ad2810981
299 3232fa7938cb8aca6deb3bc35a955ecdb86628e67cc2020c7bcc8c70253f9440
291 3bc33c6ed972f3321d81bb787550b0a76f91cb5212496e4121d7ce4b66602f06
291 51623e4bad616b26cacd90b002178365c12174c3d375dd897b3af68b6254a0a4
291 57ca444c9da946bde2e8e7aada01443224d6388e29fcadea87a97ce482e67061
291 5b85e758ce686fcf16e6107b42985317d9f11977894ee00d6c33c473733d1857
292 65b987c3feec0578f6660718a9d809e5ad15deda9bc680b5e86d5a2fce513e1b
292 d6aeecb45b9d2998c0e3b1ef761feea0541bdc549708c763cc40bee719743bb0

Those checksums seem to be created for all files, this is the full file listing of the example repo

$ rclone ls artifactory:/
155 restic/config
64 restic/config.sha256
213 restic/snapshots/18977eeeeed126528312aabc0ae0293af91fa04fb53a7d79db3c2e4b2e09893f
64 restic/snapshots/18977eeeeed126528312aabc0ae0293af91fa04fb53a7d79db3c2e4b2e09893f.sha256
447 restic/keys/403595b9f36a976e9a01e0b4ed5732f05c33099a5d660ccaf75509f58c5833c2
64 restic/keys/403595b9f36a976e9a01e0b4ed5732f05c33099a5d660ccaf75509f58c5833c2.sha256
155 restic/locks/72e6921f9ba1ec878468cb71ed8447ae9dd29e2b4d3ef9d7f2eb7c05703bb8d2
64 restic/locks/72e6921f9ba1ec878468cb71ed8447ae9dd29e2b4d3ef9d7f2eb7c05703bb8d2.sha256
366 restic/index/6585260a8e61a24bccd82c6eb892486f1a8769e467a638455d633c1af64ce1be
64 restic/index/6585260a8e61a24bccd82c6eb892486f1a8769e467a638455d633c1af64ce1be.sha256
643 restic/data/8c/8c467e3741a3178b96d85b8ef3e2b41a798f83c9ff069fdb8b09054cb96567ed
64 restic/data/8c/8c467e3741a3178b96d85b8ef3e2b41a798f83c9ff069fdb8b09054cb96567ed.sha256
1512 restic/data/2d/2dcb669f18ece903c80ff0eba3090ad8c73972588a251b2228895fc78b42dd35
64 restic/data/2d/2dcb669f18ece903c80ff0eba3090ad8c73972588a251b2228895fc78b42dd35.sha256

I don’t see that I can disable sha256 file generation, would it be an option to adjust the file matching patterns in restic to only match files without endings? I guess that would resolve my issue.

20

Honestly, I think you should first of all ask the Artifactory people how to disable the “feature” where their software creates unwanted files out of the blue in your storage. It’s not very nice of it to do so. At the very least it should be configurable.