Hey there… i am using exact this method… thx to this thread!
everything works so far… but is slow as hell…
Big machine, from hetzner to hetzner storagebox… the networkinterface and CPU are nearly idling.
Is there any kind of throttling? is rclone serve restic --stdio
generaly slow/a bottleneck?
Since check --read-data
constantly transfers Data between restic-machine and storagebox, i assumed that either the CPU/RAM would be busy or the transferrate would be high… but the fact that both are at low level, confuses me.
edit:
Additional info:
System: Hetzner, AX51-NVMe (AMD Ryzen 7 3700X, 64GB RAM)
Load average: 0.96 1.24 1.09
(While check --read-data
)
$ sudo /usr/local/bin/restic version
restic 0.16.4 compiled with go1.21.6 on linux/amd64
# sudo /usr/local/bin/restic -o rclone.program=ssh -p23 uxxxxx-subx@uxxxxx-subx.your-storagebox.de forced-command -r rclone: check --read-data
using temporary cache in /tmp/restic-check-cache-1360487135
rclone: 2024/03/14 06:00:42 DEBUG : rclone: Version "v1.63.1" starting with parameters ["/.xxxxxxxxxxx/rclone" "serve" "restic" "--stdio" "--append-only" "-vv" "."]
rclone: 2024/03/14 06:00:42 DEBUG : Creating backend with remote "."
rclone: 2024/03/14 06:00:42 DEBUG : Using config file from "/home/.config/rclone/rclone.conf"
rclone: 2024/03/14 06:00:42 DEBUG : fs cache: renaming cache item "." to be canonical "/home"
rclone: 2024/03/14 06:00:42 DEBUG : file-17096857651005817315: GET request error: object not found
rclone: 2024/03/14 06:00:42 DEBUG : keys: list request
create exclusive lock for repository
rclone: 2024/03/14 06:00:43 DEBUG : locks: list request
rclone: 2024/03/14 06:00:43 DEBUG : locks: list request
rclone: 2024/03/14 06:00:43 DEBUG : snapshots: list request
load indexes
rclone: 2024/03/14 06:00:43 DEBUG : index: list request
check all packs
rclone: 2024/03/14 06:01:24 DEBUG : data: list request
check snapshots, trees and blobs
rclone: 2024/03/14 06:01:42 INFO :
rclone: Transferred: 796.982 MiB / 801.041 MiB, 99%, 12.855 MiB/s, ETA 0s
rclone: Transferred: 339 / 341, 99%
rclone: Elapsed time: 1m0.0s
rclone: Transferring:
rclone: * data/d3/d3584e665716c0…c6a972d3a9ca130660fc90: 80% /4.068Mi, 0/s, -
rclone: * data/ce/ce00d7b2d3cf3e…baf0963e32a04d70f8302d: 18% /4.021Mi, 0/s, -
rclone:
rclone: 2024/03/14 06:02:42 INFO :
rclone: Transferred: 1.695 GiB / 1.760 GiB, 96%, 12.979 MiB/s, ETA 5s
rclone: Transferred: 474 / 478, 99%
rclone: Elapsed time: 2m0.0s
rclone: Transferring:
rclone: * data/83/830e5c06075c55…d6e1edc71b50bacf94a495: 83% /61.928Mi, 1.319Mi/s, 7s
rclone: * data/bb/bb0495ca8588d5…5eb560bd615990b6c2cee2: 51% /16.228Mi, 4.187Mi/s, 1s
rclone: * data/dc/dc14f2a86a3eb1…954de44f332373f2923e64: 14% /52.411Mi, 5.776Mi/s, 7s
rclone: * data/d7/d7bad5b646d761…46ae13fae77d3d841303b7: 25% /4.041Mi, 0/s, -
rclone:
rclone: 2024/03/14 06:03:42 INFO :
rclone: Transferred: 2.778 GiB / 2.789 GiB, 100%, 12.979 MiB/s, ETA 0s
rclone: Transferred: 707 / 712, 99%
rclone: Elapsed time: 3m0.0s
rclone: Transferring:
rclone: * data/1b/1b1a82a7ab87e4…b61e53a999a6debe87f3c3: 81% /4.452Mi, 0/s, -
rclone: * data/7e/7ea6d0eec6d239…69a99b6f93fb4b00ff1f2a: 91% /4.830Mi, 0/s, -
rclone: * data/b2/b2e0481e6a5d71…d293e1f9b99c52df19ddb6: 17% /4.779Mi, 0/s, -
rclone: * data/8f/8fb47326039e1b…7b6ba9f8c96706f44684dc: 27% /4.161Mi, 0/s, -
rclone: * data/0a/0a921c53caa4a4…094c6d05201fe112be5214: 30% /4.511Mi, 0/s, -
rclone:
rclone: 2024/03/14 06:04:42 INFO :
rclone: Transferred: 4.163 GiB / 4.168 GiB, 100%, 12.979 MiB/s, ETA 0s
rclone: Transferred: 1081 / 1086, 100%
rclone: Elapsed time: 4m0.0s
rclone: Transferring:
rclone: * data/a4/a4695ab5f28218…a90e5a582d658b835e569c: 90% /4.040Mi, 2.342Mi/s, 0s
rclone: * data/aa/aa4e591e3cbcb8…a63680def7f60101168f80: 75% /4.099Mi, 0/s, -
rclone: * data/88/88a82d9a045af4…995addf021c9ecd2471982: 40% /4.100Mi, 0/s, -
rclone: * data/9d/9d52714401edd7…c607e09791221371de20b5: 75% /4.114Mi, 0/s, -
rclone: * data/95/954d79ece5a25c…ae7cc15099c617803377b3: 96% /4.099Mi, 0/s, -
rclone:
rclone: 2024/03/14 06:05:42 INFO :
rclone: Transferred: 5.294 GiB / 5.294 GiB, 100%, 12.979 MiB/s, ETA 0s
rclone: Transferred: 1381 / 1381, 100%
rclone: Elapsed time: 5m0.0s
rclone:
rclone: 2024/03/14 06:06:42 INFO :
rclone: Transferred: 6.369 GiB / 6.386 GiB, 100%, 12.979 MiB/s, ETA 1s
rclone: Transferred: 1662 / 1667, 100%
rclone: Elapsed time: 6m0.0s
rclone: Transferring:
rclone: * data/84/849b950211fd24…f40c2fb4c4785617d8a871: 12% /4.048Mi, 0/s, -
rclone: * data/08/08e8d22a2bd8c1…803e1a05495010a1ba583b: 13% /4.052Mi, 0/s, -
rclone: * data/74/74bafcc737a365…0806727e8f7b1f93bfb2c1: 15% /4.122Mi, 0/s, -
rclone: * data/bb/bb67fa3a10fc2d…057e69dafc145eb3e5f823: 31% /4.082Mi, 0/s, -
rclone: * data/4a/4a5cfda1535e10…40fb3dc3615faf0bced2ea: 21% /4.039Mi, 0/s, -
rclone:
rclone: 2024/03/14 06:07:42 INFO :
rclone: Transferred: 7.261 GiB / 7.272 GiB, 100%, 12.979 MiB/s, ETA 0s
rclone: Transferred: 1906 / 1910, 100%
rclone: Elapsed time: 7m0.0s
rclone: Transferring:
rclone: * data/e5/e53358db985929…2a0f1834cb663a68468367: 15% /4.041Mi, 0/s, -
rclone: * data/fd/fdd14a95a05752…db111db8abdec47dd3b2e9: 22% /4.075Mi, 0/s, -
rclone: * data/44/445ec19ef84e64…8b9e175fd36c6d9f58c4ba: 24% /4.133Mi, 0/s, -
rclone: * data/19/19926d6bc285e2…f4c43aca4f2f1e77d5d20a: 47% /3.235Mi, 0/s, -
rclone:
[6:46] 100.00% 133 / 133 snapshots
read all data
rclone: 2024/03/14 06:08:42 INFO :
rclone: Transferred: 8.200 GiB / 8.216 GiB, 100%, 12.979 MiB/s, ETA 1s
rclone: Transferred: 2148 / 2152, 100%
rclone: Elapsed time: 8m0.0s
rclone: Transferring:
rclone: * data/60/609d38e55e2b9b…0e81ba6db587e5b435827d: 98% /5.814Mi, 5.091Mi/s, 0s
rclone: * data/20/20fea539238e3d…0d7b97105190d493b1d52b: 33% /17.645Mi, 5.624Mi/s, 2s
rclone: * data/10/10254b99d9afa6…d707d969bcc3f8e0844dbf: 82% /4.147Mi, 0/s, -
rclone: * data/30/30997f391d504d…7324e6d3f767a8a1eb8ec8: 37% /5.964Mi, 0/s, -
rclone:
rclone: 2024/03/14 06:09:42 INFO :
rclone: Transferred: 9.791 GiB / 9.802 GiB, 100%, 12.979 MiB/s, ETA 0s
rclone: Transferred: 2504 / 2508, 100%
rclone: Elapsed time: 9m0.0s
rclone: Transferring:
rclone: * data/f0/f0847d50815db3…be8c0218c852abd74f8172: 35% /4.001Mi, 0/s, -
rclone: * data/e0/e08bc2d7dc51f3…711b37af6179b4372b89aa: 34% /5.835Mi, 0/s, -
rclone: * data/30/30ff959ed10c78…0fc0e458e91e8aedd0b229: 69% /4.008Mi, 0/s, -
rclone: * data/10/105560c4819d22…b74c55d42046b3cfc08005: 22% /4.628Mi, 0/s, -
rclone:
rclone: 2024/03/14 06:10:42 INFO :
rclone: Transferred: 11.361 GiB / 11.371 GiB, 100%, 12.979 MiB/s, ETA 0s
rclone: Transferred: 2845 / 2850, 100%
rclone: Elapsed time: 10m0.0s
rclone: Transferring:
rclone: * data/f0/f047cc8277b74d…6ccc57c75ae845a5840a90: 86% /4.423Mi, 0/s, -
rclone: * data/30/302874d4196789…6c701daf92a25d63d8f497: 84% /5.912Mi, 0/s, -
rclone: * data/c0/c083332b95fe38…85fe854ff19956ec18367b: 31% /4.016Mi, 0/s, -
rclone: * data/d0/d027701a92f450…6b9cc2aa271e0ffcdd0c70: 42% /4.795Mi, 0/s, -
rclone: * data/a0/a0a9cb6c50d483…bc9ba5b5bcfc92915cd91c: 46% /5.056Mi, 0/s, -
rclone:
and so on...
any idea on how to improv speed? what is the bottleneck? how to debug?