Which protocol should I choose for remote Linux backups?

Fair point, I am actually wondering, if that is a bug in their WebDav implementation.
If I use e.g. their native web ui browser, the checksum files are hidden.

Seems a bit like WebDav leaks some internal objects in comparison.

I agree. Perhaps itā€™s as simple as them storing the checksum in a file, but hiding it here and there, and missed doing so over WebDAV. Ask them? :slight_smile:

Yes, Iā€™ll ask what their take on that is, but it seem to be like that for quite a while.
I found [RTFACT-16062] Enhance Artifactory's WebDAV capabilities - JFrog JIRA from 2018 which mentioned that files as well and asking for improvements. There was no response yet whatsoeverā€¦

Thatā€™s totally it. Add a comment to it if you can, asking them to get this fixed? :smiley:

I replied in [RTFACT-16062] Enhance Artifactory's WebDAV capabilities - JFrog JIRA, :grinning: :crossed_fingers:

Thanks. I think restic might be able to work around this otherwise, but itā€™s something that has to be implemented of course. The best thing would be if services like that didnā€™t misbehave.

I had the discussion in rustic how to treat extra files (which donā€™t match the hex pattern of a SHA256) as some users did add commentary files (like a README which explains the data structure).
So I decided to simply ignore any file not fitting in the pattern. But IIRC that was only with the local backend.

I totally understand, every patch for a particular target is something which needs to be maintained over time and adds mental load for developers. Especially, because it is not really an issue with WebDAV, other WebDAV servers work just fine. I am not sure, if Artifactory will change its current implementation as it would be a breaking change, even if it was introduced by accident initially.
On the other hand, Artifactory is a repository hosting service, they could also support a restic type of repository and implement resticā€™s REST API directly (API Docs) . I guess it would be easier to implement in comparison to docker or other repository schemas :sweat_smile:
This might also help to adopt restic in more enterprise environments, as Artifactory usually is configured with real certificates and already needs to be secured.

Ah, okay, good to know, thanks Alex. I am not familiar with resticā€™s code structure, but would it make sense to have that detection ported to remote repository formats than as well? I mean it would not only workaround the Artifactory issue, but might be more consistent and also prevents issues, if someone copies existing repositories with those additional files to a remote server and tries to restore e.g. via sftp.

@alexweiss dit not write ā€œresticā€. He wrote ā€œrusticā€, which is another backup program. ā€œrusticā€ and ā€œresticā€ does not match one another 1:1 when it comes to features. See GitHub - rustic-rs/rustic: rustic - fast, encrypted, and deduplicated backups powered by Rust for more information. From the top there:

Rustic is a backup tool that provides fast, encrypted, deduplicated backups. It reads and writes the restic repo format desribed in the design document and can therefore be used as a complete replacement for restic.

Thanks for clarifying Martin, I was obviously mixing things up.
I was trying to find that place and was wondering where in restic/id.go at v0.14.0 Ā· restic/restic Ā· GitHub or on master restic/id.go at master Ā· restic/restic Ā· GitHub the check would be implemented, that explains it :smiley:.

In fact, I was referring to these lines.

But looking back, I donā€™t like much how itā€™s implemented and it is only implemented for local backends so far. Iā€™ll change it such that the behavior is implemented and identical for all backends, thanks for the hint!

Sorry to hijack this thread, but I must say that Iā€™m very unhappy with the name, the strings restic and rustic are too similar, so users will be confused :frowning:

@alexweiss please consider using a name that has a higher hamming distance

4 Likes
2 Likes

Iā€™m not a code contributor; purely a daily restic user, with a deep respect for people who put their time and passion into great open-source products such as restic. Especially respect for those people who start such endeavors.

There are only so many ways we can show our appreciation to creators such as @fd0 - they donā€™t ask much of us; not even our money.

@alexweiss , that is now at least the second time @fd0 has politely asked something very reasonable of you. I hope you have enough decency to do the right thing, and show him the respect that he deserves to receive from everyone in this community.

5 Likes

Thanks a lot for the code update @MichaelEischer and @fd0 :slight_smile:

@fd0 I made a consideration. In my opinion rustic is a pretty good name for a backup tool that is written in rust. Moreover, Iā€™d rather invest my time in improving software than in a name change I donā€™t see the benefit of. I am pretty sure that you as a maintainer can understand that.

Thanks for your praise! I am totally with you here. You are very right that those people more often face barriers and unpleasant discussions than a thank-you. I do hope that your respect is not restricted to a single software project but also to others who just started.

1 Like

Just wanted to confirm that the fix worked like a charm :slightly_smiling_face::tada:, I used the previously created repo and tried the manually compiled binary from master

$ ./restic version
restic 0.14.0-dev (compiled manually) compiled with go1.19.1 on linux/amd64

$ ./restic snapshots -r rclone:artifactory:restic
enter password for repository:
repository 23ae2878 opened (repository version 2) successfully, password is correct
ID        Time                 Host        Tags        Paths
---------------------------------------------------------------------------------
18977eee  2022-09-19 08:26:35  xxx                    /tmp/restic-init-debug.log
---------------------------------------------------------------------------------
1 snapshots

$ ./restic restore 18977eee --target restore -r rclone:artifactory:restic
enter password for repository:
repository 23ae2878 opened (repository version 2) successfully, password is correct
restoring <Snapshot 18977eee of [/tmp/restic-init-debug.log] at 2022-09-19 08:26:35.880118336 +0200 CEST by xxx@xxx> to restore

$ find restore/
restore/
restore/tmp
restore/tmp/restic-init-debug.log

$ ./restic forget 18977eee -r rclone:artifactory:restic
enter password for repository:
repository 23ae2878 opened (repository version 2) successfully, password is correct
[0:00] 100.00%  1 / 1 files deleted

Thanks so much, I did not hear anything from the Artifactory guys so far, but this would make their fix less important.

Just out of curiosity, how are releases for restic planned? Are you releasing on a schedule or when a certain amount of changes were merged?

2 Likes

There is no fixed schedule. A new release is usually made after adding a decent amount of changes and some related polishing. But thatā€™s pretty hard to predict when that is the case. In particular, when adding large new features, we wait some time before a new release, to ensure that we havenā€™t missed something important.

1 Like

Sounds good, thanks