I am attempting to restore one .sql file from a Wasabi S3 hosted bucket, it’s approximately 16GB in size. I tried restoring using the standard restic implementation and now I’m testing using rclone. I previously restored files without any issues but now when I try to restore it takes ages. I left this restore running over last few days and it seems to have only restored around ~8GB of it.
I’m running CentOS 7, with the latest build of restic restic 0.9.6 compiled with go1.13.4 on linux/amd64 (from source). When I try to restore via rclone I get some logs indicating that my restore is going at around 10-25 kBytes/s. Also each time I perform a backup I prune at the end.
Any tips as to what may be causing this dip in performance?
After further lurking the forums for a solution I noticed a comment which mentioned that SMR HDD could impact performance, however this doesn’t seem to be the case for me. I decided it would be worth a shot restoring on a machine with an SSD instead and got the following results:
Which seems to have slightly improved my performance, but I have a gigabit connection so I was expecting a much better performance. When running ‘top’ it seems that restic is working hard in the background as well with a 100% CPU core usage. Has anyone had a similar sort of issue? My bucket has around 40k+ snapshots.
I’m not sure if it was related to this, as the speed of my restore was incredibly slow through out the entire restore run while this issue mentions that it progressively slows down. It seems that the fix for this was already merged into master and I tried running a fresh build of master which didn’t seem to resolve the issue. However, when I was lurking yesterday I came across:
And while this issue has already been merged as well it gave me an idea to try the latest beta binary version (just like the person in the issue you mentioned). I used: https://beta.restic.net/restic-v0.9.6-377-gc34c7316/ which seemed to resolve the issue I was having.
Is there a public beta branch for restic? or is that only accessible by devs?
But does it also contain any changes that are present on the beta releases? I was slightly confused as I couldn’t figure out what the version on the beta release related to in terms of commits. I went into the commit history of master branch to see if I can find a version relating to:
But could not find anything relating to this, which lead me to believe there may have been a private Beta repo which gets merged into master, and those version relate directly to the Beta branch.
Do you know by any chance how the versioning there works?
I also compiled the latest master branch along the way and tested my restore using that which seemed to still operate slowly, in hindsight I should’ve deleted any previous version prior to testing, just in case it was still using the old version. I’m gonna try that and report back, may have just been me being an idiot this whole time.
It’s simply a direct build of the master branch. So everything that’s been merged will be in it.
If you grab the commit hash from the last part of the filename, but without the initial g, in your case the result would be c34c7316, you can look for that commit hash in the restic repository, e.g. https://github.com/restic/restic/commit/c34c7316 .
I have no idea what that g is about, maybe @fd0 can tell us?