Thanks for the reply, so i was heading a lot of thought and found something very odd i dont know why restic was backing up also in another folder it was very odd so what i did is create another repo so far everything seems good, might of been that issue
Iâm pretty confident that this issue is caused by the underlying file system, which seems to mix up 32 bit and 64 bit integers. It tries to return a negative 32 bit value, which is interpreted as a 64 bit integer, so it becomes positive. All the behavior we see is explained by my hypothesis.
@killmasta93 can you tell us which file system youâre using? Do you run restic on a NAS or similar embedded device?
thanks for the reply, currently running restic on proxmox debian 9 which cifs mounts the windows smb share then also mount the NAS share and uses restic on proxmox to backup, so the issue could be the NAS?
Just to make sure I understand your setup correctly: The backup source is on a CIFS share and the backup repository is stored on the host itself/the NAS?
Could you test my proposed modifications to the debug-chunker branch? These should be enough to tell which file is the culprit:
Thanks for the reply, so what i did is change the backup source is on a CIFS share but changed the backup repo to another server to see if that was the issue so i created a new repo on proxmox host which the first 2 backup works
but then i got this
To test my modifications you need to checkout the debug-chunker branch, modifie the line I asked you to, then build the restic binary and use that binary for the next backup runs.
git clone https://github.com/restic/restic
cd restic/cmd/restic
git checkout debug-chunker
go build
then edit
nano restic/chunker/chunker.go
changed the file
//if n < 0 {
if n > cap(buf) {
return Chunk{}, fmt.Errorf("ReadFull returned negative number of bytes read: %v", n)
}
and reran rescript and for some odd reason this time it worked i guess im going to wait tomorrow and run again
this is the output
root@prometheus2:/media# sh fileserver.sh
mount.cifs kernel mount options: ip=192.168.3.209,unc=\\192.168.3.209\discod$,ve rs=2.1,nouser_xattr,user=administrador,pass=********
================================================================================
STARTING SCRIPT
================================================================================
Date and Time: Sun May 17 2020 09:23:16 PM
System: Debian GNU/Linux 9
Hostname: prometheus2
Repository Location: /media/backupnas
Restic Version: 0.9.6-dev
--------------------------------------------------------------------------------
[Taking a Snapshot...]
open repository
lock repository
load index files
using parent snapshot 7dc9819a
start scan on [/media/servers/ad/shares]
start backup on [/media/servers/ad/shares]
scan: Readdirnames /media/servers/ad/shares/System Volume Information failed: readdirent: permission denied
scan: Readdirnames /media/servers/ad/shares/Ventas/ropa/GW BIKE APPAREL - VENTAS INTERNACIONALES/29-01-2019/ROPA/PROVEEDORES/JQ/ORDERS/GAFAS/ORDER 2019/PLANTILLAS/FINAL
scan finished in 1911.467s: 1496565 files, 748.251 GiB
uploaded intermediate index cd339de5
error: open /media/servers/ad/shares/Ingenieria/Carlos.Rodriguez/Inventor/MPOS1.0/Imported Components/jrb006-480-carbon-/ÂŁíż6.6.ipt: no such file or directory
uploaded intermediate index 83475287
error: open /media/servers/ad/shares/Recursos Humanos/blanca.gonzalez/HA BICICLETAS/GESTION DE CALIDAD/PROGRAMAS/PSV/Cartilla Regional Antioquiadd/.DS_Store: permission
error: open /media/servers/ad/shares/Recursos Humanos/blanca.gonzalez/HA BICICLETAS/GESTION DE CALIDAD/PROGRAMAS/PSV/Cartilla Regional Antioquiadd/INSTRUCTIVO PARA REAission denied
error: open /media/servers/ad/shares/Recursos Humanos/blanca.gonzalez/HA BICICLETAS/GESTION DE CALIDAD/PROGRAMAS/PSV/Cartilla Regional Antioquiadd/Instructivo varios te
error: open /media/servers/ad/shares/Recursos Humanos/blanca.gonzalez/HA BICICLETAS/GESTION DE CALIDAD/PROGRAMAS/PSV/Cartilla Regional Antioquiadd/PDF ANTIOQUIA EJE CAF
error: open /media/servers/ad/shares/Recursos Humanos/blanca.gonzalez/HA BICICLETAS/GESTION DE CALIDAD/PROGRAMAS/PSV/Cartilla Regional Antioquiadd/cartilla Regional ant
error: open /media/servers/ad/shares/Recursos Humanos/blanca.gonzalez/HA BICICLETAS/GESTION DE CALIDAD/PROGRAMAS/PSV/Cartilla Regional Antioquiadd/generar certificados
error: open /media/servers/ad/shares/Recursos Humanos/compartido/GH-MG-01 Manual de Funciones/GESTION DE CALIDAD/PROGRAMAS/PSV/Cartilla Regional Antioquia/.DS_Store: p
error: open /media/servers/ad/shares/Recursos Humanos/compartido/GH-MG-01 Manual de Funciones/GESTION DE CALIDAD/PROGRAMAS/PSV/Cartilla Regional Antioquia/INSTRUCTIVO ocx: permission denied
error: open /media/servers/ad/shares/Recursos Humanos/compartido/GH-MG-01 Manual de Funciones/GESTION DE CALIDAD/PROGRAMAS/PSV/Cartilla Regional Antioquia/Instructivo
error: open /media/servers/ad/shares/Recursos Humanos/compartido/GH-MG-01 Manual de Funciones/GESTION DE CALIDAD/PROGRAMAS/PSV/Cartilla Regional Antioquia/PDF ANTIOQUI
error: open /media/servers/ad/shares/Recursos Humanos/compartido/GH-MG-01 Manual de Funciones/GESTION DE CALIDAD/PROGRAMAS/PSV/Cartilla Regional Antioquia/cartilla Reg
error: open /media/servers/ad/shares/Recursos Humanos/compartido/GH-MG-01 Manual de Funciones/GESTION DE CALIDAD/PROGRAMAS/PSV/Cartilla Regional Antioquia/generar cert
error: Readdirnames /media/servers/ad/shares/System Volume Information failed: readdirent: permission denied
uploaded intermediate index 24c752d7
error: open /media/servers/ad/shares/Ventas/ropa/GW BIKE APPAREL - VENTAS INTERNACIONALES/29-01-2019/ROPA/PROVEEDORES/JQ/ORDERS/GAFAS/ORDER 2019/PLANTILLAS/FINAL/Gafas
error: open /media/servers/ad/shares/Ventas/ropa/GW BIKE APPAREL - VENTAS INTERNACIONALES/29-01-2019/ROPA/PROVEEDORES/JQ/ORDERS/GAFAS/ORDER 2019/PLANTILLAS/FINAL/Gafas
error: open /media/servers/ad/shares/Ventas/ropa/GW BIKE APPAREL - VENTAS INTERNACIONALES/29-01-2019/ROPA/PROVEEDORES/JQ/ORDERS/GAFAS/ORDER 2019/PLANTILLAS/FINAL/Gafas
error: open /media/servers/ad/shares/Ventas/ropa/GW BIKE APPAREL - VENTAS INTERNACIONALES/29-01-2019/ROPA/PROVEEDORES/JQ/ORDERS/GAFAS/ORDER 2019/PLANTILLAS/FINAL/Gafas
error: open /media/servers/ad/shares/Ventas/ropa/GW BIKE APPAREL - VENTAS INTERNACIONALES/29-01-2019/ROPA/PROVEEDORES/JQ/ORDERS/GAFAS/ORDER 2019/PLANTILLAS/FINAL/Gafas
error: open /media/servers/ad/shares/Ventas/ropa/GW BIKE APPAREL - VENTAS INTERNACIONALES/29-01-2019/ROPA/PROVEEDORES/JQ/ORDERS/GAFAS/ORDER 2019/PLANTILLAS/FINAL/Gafas
error: open /media/servers/ad/shares/Ventas/ropa/GW BIKE APPAREL - VENTAS INTERNACIONALES/29-01-2019/ROPA/PROVEEDORES/JQ/ORDERS/GAFAS/ORDER 2019/PLANTILLAS/FINAL/Gafas
error: open /media/servers/ad/shares/Ventas/ropa/GW BIKE APPAREL - VENTAS INTERNACIONALES/29-01-2019/ROPA/PROVEEDORES/JQ/ORDERS/GAFAS/ORDER 2019/PLANTILLAS/FINAL/Gafas
error: open /media/servers/ad/shares/Ventas/ropa/GW BIKE APPAREL - VENTAS INTERNACIONALES/29-01-2019/ROPA/PROVEEDORES/JQ/ORDERS/GAFAS/ORDER 2019/PLANTILLAS/FINAL/Gafas
error: open /media/servers/ad/shares/Ventas/ropa/GW BIKE APPAREL - VENTAS INTERNACIONALES/29-01-2019/ROPA/PROVEEDORES/JQ/ORDERS/GAFAS/ORDER 2019/PLANTILLAS/FINAL/Gafas
error: open /media/servers/ad/shares/Ventas/ropa/GW BIKE APPAREL - VENTAS INTERNACIONALES/29-01-2019/ROPA/PROVEEDORES/JQ/ORDERS/GAFAS/ORDER 2019/PLANTILLAS/FINAL/Gafas
error: open /media/servers/ad/shares/Ventas/ropa/GW BIKE APPAREL - VENTAS INTERNACIONALES/29-01-2019/ROPA/PROVEEDORES/JQ/ORDERS/GAFAS/ORDER 2019/PLANTILLAS/FINAL/Gafas
error: Readdirnames /media/servers/ad/shares/Ventas/ropa/GW BIKE APPAREL - VENTAS INTERNACIONALES/29-01-2019/ROPA/PROVEEDORES/JQ/ORDERS/GAFAS/ORDER 2019/PLANTILLAS/FINA
uploaded intermediate index 3ace46a5
Files: 115 new, 135 changed, 1496289 unmodified
Dirs: 0 new, 3 changed, 0 unmodified
Data Blobs: 769 new
Tree Blobs: 4 new
Added to the repo: 787.030 MiB
processed 1496539 files, 748.183 GiB in 1:57:42
snapshot 7ee2f519 saved
There are 6 exclusion rules...
--------------------------------------------------------------------------------
[Snapshots List...]
ID Time Host Tags
--------------------------------------------------
35c5748f 2020-05-14 22:32:20 prometheus2
7dc9819a 2020-05-15 18:46:44 prometheus2
7ee2f519 2020-05-17 21:23:16 prometheus2
--------------------------------------------------
3 snapshots
--------------------------------------------------------------------------------
Next cleanup and check in 4 days, 9 hours and 41 minutes...
--------------------------------------------------------------------------------
Summarized Info Restore Size Deduplicated Size
--------------------------------------------------------------------------------
Latest Snapshot 748.183 GiB 517.261 GiB
All Snapshots 748.636 GiB 519.056 GiB
--------------------------------------------------------------------------------
End: Sun May 17 2020 11:26:09 PM
Duration: 2 hours, 2 minutes and 53 seconds
================================================================================
SCRIPT ENDED
================================================================================
=============================
Sending Output to Zabbix
=============================
[Running discovery of rescript repos]
info from server: "processed: 1; failed: 0; total: 1; seconds spent: 0.000073"
sent: 1; skipped: 0; total: 1
Extracting from: /root/.rescript/logs/fileserver-log-2020-05-17-21:23.log
Repository: fileserver
Checksum: 562253777
[Capture Restic Backup]
Snapshot ID: 7ee2f519
Bytes Added: 825260769
Time Processed: 7062
Bytes Processed: 7062
Are you sure that youâve used that binary that includes the additional chunker.go change? We previously saw exactly the same crash for if n < 0 {. And as that variant of the check didnât match previously, if n > cap(buf) should match instead and result in a different error.
That procedure seems to be correct (assuming that you copied/moved the newly built binary to wherever it needs to be in order to be used by the backup script).
Could you check again whether the chunker.go file contains the modification and whether for _, b := range buf[c.bpos:c.bmax] { is at line 303? I just want to be sure that the backup run did not accidentally use an older binary.
Thank you so much for the reply, this is what i got, but whats funny so far its been 4 days and the issue has not happened again the only thing odd is that the backup takes like 11 hours which i dont know why this is the log
mount.cifs kernel mount options: ip=192.168.3.209,unc=\\192.168.3.209\discod$,vers=2.1,nouser_xattr,user=administrador,pass=********
================================================================================
STARTING SCRIPT
================================================================================
e[33mDate and Time:e[0m Fri May 22 2020 06:10:01 PM
e[33mSystem:e[0m Debian GNU/Linux 9
e[33mHostname:e[0m prometheus5
e[33mRepository Location:e[0m /media/bakfileserver
e[33mRestic Version:e[0m 0.9.6-dev
--------------------------------------------------------------------------------
e[33m[Taking a Snapshot...]e[0m
open repository
lock repository
load index files
using parent snapshot d06af92e
start scan on [/media/servers/ad/shares]
start backup on [/media/servers/ad/shares]
scan: Readdirnames /media/servers/ad/shares/System Volume Information failed: readdirent: permission denied
scan: Readdirnames /media/servers/ad/shares/Ventas/ropa/GW BIKE APPAREL - VENTAS INTERNACIONALES/29-01-2019/ROPA/PROVEEDORES/JQ/ORDERS/GAFAS/ORDER 2019/PLANTILLAS/FINAL/Gafas GW /CAMBIOS failed: readdirent: no such file or directory
scan finished in 12843.854s: 1498617 files, 750.396 GiB
uploaded intermediate index b0ff60c8
uploaded intermediate index c211994d
error: lstat /media/servers/ad/shares/Comercio Exterior/edwin.calle/Outlook/~archive.pst.tmp: no such file or directory
uploaded intermediate index 261428bc
uploaded intermediate index b26b68be
uploaded intermediate index 08629b31
uploaded intermediate index e6552a77
uploaded intermediate index 7edd7c03
error: open /media/servers/ad/shares/Ingenieria/Carlos.Rodriguez/Inventor/MPOS1.0/Imported Components/jrb006-480-carbon-/ÂŁíż6.6.ipt: no such file or directory
uploaded intermediate index 4911bc82
uploaded intermediate index 1de93930
error: open /media/servers/ad/shares/Recursos Humanos/blanca.gonzalez/HA/GESTION DE CALIDAD/PROGRAMAS/PSV/Cartilla Regional Antioquiadd/.DS_Store: permission denied
error: open /media/servers/ad/shares/Recursos Humanos/blanca.gonzalez/HA/GESTION DE CALIDAD/PROGRAMAS/PSV/Cartilla Regional Antioquiadd/INSTRUCTIVO PARA REALIZR LAS INSCRIPCIONES POR SERVICIOS EN LINEA VERSION 2016.docx: permission denied
error: open /media/servers/ad/shares/Recursos Humanos/blanca.gonzalez/HA/GESTION DE CALIDAD/PROGRAMAS/PSV/Cartilla Regional Antioquiadd/Instructivo varios temas para un trabajador.pdf: permission denied
error: open /media/servers/ad/shares/Recursos Humanos/blanca.gonzalez/HA/GESTION DE CALIDAD/PROGRAMAS/PSV/Cartilla Regional Antioquiadd/PDF ANTIOQUIA EJE CAFETERO 20163.pdf: permission denied
error: open /media/servers/ad/shares/Recursos Humanos/blanca.gonzalez/HA/GESTION DE CALIDAD/PROGRAMAS/PSV/Cartilla Regional Antioquiadd/cartilla Regional antioquia.xls: permission denied
error: open /media/servers/ad/shares/Recursos Humanos/blanca.gonzalez/HA/GESTION DE CALIDAD/PROGRAMAS/PSV/Cartilla Regional Antioquiadd/generar certificados 2016.docx: permission denied
error: open /media/servers/ad/shares/Recursos Humanos/compartido/GH-MG-01 Manual de Funciones/GESTION DE CALIDAD/PROGRAMAS/PSV/Cartilla Regional Antioquia/.DS_Store: permission denied
error: open /media/servers/ad/shares/Recursos Humanos/compartido/GH-MG-01 Manual de Funciones/GESTION DE CALIDAD/PROGRAMAS/PSV/Cartilla Regional Antioquia/INSTRUCTIVO PARA REALIZR LAS INSCRIPCIONES POR SERVICIOS EN LINEA VERSION 2016.docx: permission denied
error: open /media/servers/ad/shares/Recursos Humanos/compartido/GH-MG-01 Manual de Funciones/GESTION DE CALIDAD/PROGRAMAS/PSV/Cartilla Regional Antioquia/Instructivo varios temas para un trabajador.pdf: permission denied
error: open /media/servers/ad/shares/Recursos Humanos/compartido/GH-MG-01 Manual de Funciones/GESTION DE CALIDAD/PROGRAMAS/PSV/Cartilla Regional Antioquia/PDF ANTIOQUIA EJE CAFETERO 2016.pdf: permission denied
error: open /media/servers/ad/shares/Recursos Humanos/compartido/GH-MG-01 Manual de Funciones/GESTION DE CALIDAD/PROGRAMAS/PSV/Cartilla Regional Antioquia/cartilla Regional antioquia.xls: permission denied
error: open /media/servers/ad/shares/Recursos Humanos/compartido/GH-MG-01 Manual de Funciones/GESTION DE CALIDAD/PROGRAMAS/PSV/Cartilla Regional Antioquia/generar certificados 2016.docx: permission denied
uploaded intermediate index 35ede91c
error: Readdirnames /media/servers/ad/shares/System Volume Information failed: readdirent: permission denied
uploaded intermediate index 8f107a2b
uploaded intermediate index bdbcf89d
uploaded intermediate index 64919f41
uploaded intermediate index 18572cf1
error: open /media/servers/ad/shares/Ventas/ropa/GW BIKE APPAREL - VENTAS INTERNACIONALES/29-01-2019/ROPA/PROVEEDORES/JQ/ORDERS/GAFAS/ORDER 2019/PLANTILLAS/FINAL/Gafas GW /._AF¡Caja Gafas GW.pdf: no such file or directory
error: open /media/servers/ad/shares/Ventas/ropa/GW BIKE APPAREL - VENTAS INTERNACIONALES/29-01-2019/ROPA/PROVEEDORES/JQ/ORDERS/GAFAS/ORDER 2019/PLANTILLAS/FINAL/Gafas GW /._AF¡Gafas DG CARDENAL.pdf: no such file or directory
error: open /media/servers/ad/shares/Ventas/ropa/GW BIKE APPAREL - VENTAS INTERNACIONALES/29-01-2019/ROPA/PROVEEDORES/JQ/ORDERS/GAFAS/ORDER 2019/PLANTILLAS/FINAL/Gafas GW /._AF¡Gafas DG COMANCHE.pdf: no such file or directory
error: open /media/servers/ad/shares/Ventas/ropa/GW BIKE APPAREL - VENTAS INTERNACIONALES/29-01-2019/ROPA/PROVEEDORES/JQ/ORDERS/GAFAS/ORDER 2019/PLANTILLAS/FINAL/Gafas GW /._AF¡Gafas DG CRUCEROai.pdf: no such file or directory
error: open /media/servers/ad/shares/Ventas/ropa/GW BIKE APPAREL - VENTAS INTERNACIONALES/29-01-2019/ROPA/PROVEEDORES/JQ/ORDERS/GAFAS/ORDER 2019/PLANTILLAS/FINAL/Gafas GW /._AF¡Gafas DG DESIERTO.pdf: no such file or directory
error: open /media/servers/ad/shares/Ventas/ropa/GW BIKE APPAREL - VENTAS INTERNACIONALES/29-01-2019/ROPA/PROVEEDORES/JQ/ORDERS/GAFAS/ORDER 2019/PLANTILLAS/FINAL/Gafas GW /._AF¡Gafas DG MAYA.pdf: no such file or directory
error: open /media/servers/ad/shares/Ventas/ropa/GW BIKE APPAREL - VENTAS INTERNACIONALES/29-01-2019/ROPA/PROVEEDORES/JQ/ORDERS/GAFAS/ORDER 2019/PLANTILLAS/FINAL/Gafas GW /AF¡Caja Gafas GW.pdf: no such file or directory
error: open /media/servers/ad/shares/Ventas/ropa/GW BIKE APPAREL - VENTAS INTERNACIONALES/29-01-2019/ROPA/PROVEEDORES/JQ/ORDERS/GAFAS/ORDER 2019/PLANTILLAS/FINAL/Gafas GW /AF¡Gafas DG CARDENAL.pdf: no such file or directory
error: open /media/servers/ad/shares/Ventas/ropa/GW BIKE APPAREL - VENTAS INTERNACIONALES/29-01-2019/ROPA/PROVEEDORES/JQ/ORDERS/GAFAS/ORDER 2019/PLANTILLAS/FINAL/Gafas GW /AF¡Gafas DG COMANCHE.pdf: no such file or directory
error: open /media/servers/ad/shares/Ventas/ropa/GW BIKE APPAREL - VENTAS INTERNACIONALES/29-01-2019/ROPA/PROVEEDORES/JQ/ORDERS/GAFAS/ORDER 2019/PLANTILLAS/FINAL/Gafas GW /AF¡Gafas DG CRUCEROai.pdf: no such file or directory
error: open /media/servers/ad/shares/Ventas/ropa/GW BIKE APPAREL - VENTAS INTERNACIONALES/29-01-2019/ROPA/PROVEEDORES/JQ/ORDERS/GAFAS/ORDER 2019/PLANTILLAS/FINAL/Gafas GW /AF¡Gafas DG DESIERTO.pdf: no such file or directory
error: open /media/servers/ad/shares/Ventas/ropa/GW BIKE APPAREL - VENTAS INTERNACIONALES/29-01-2019/ROPA/PROVEEDORES/JQ/ORDERS/GAFAS/ORDER 2019/PLANTILLAS/FINAL/Gafas GW /AF¡Gafas DG MAYA.pdf: no such file or directory
error: Readdirnames /media/servers/ad/shares/Ventas/ropa/GW BIKE APPAREL - VENTAS INTERNACIONALES/29-01-2019/ROPA/PROVEEDORES/JQ/ORDERS/GAFAS/ORDER 2019/PLANTILLAS/FINAL/Gafas GW /CAMBIOS failed: readdirent: no such file or directory
uploaded intermediate index 3f8b6d8e
uploaded intermediate index 3294acba
uploaded intermediate index e8d2c0df
Files: 422 new, 345 changed, 1497818 unmodified
Dirs: 0 new, 3 changed, 0 unmodified
Data Blobs: 1641 new
Tree Blobs: 4 new
Added to the repo: 1021.525 MiB
processed 1498585 files, 750.325 GiB in 11:16:04
snapshot cd496767 saved
e[33mThere are 6 exclusion rules...e[0m
--------------------------------------------------------------------------------
e[33m[Snapshots List...]e[0m
ID Time Host Tags
--------------------------------------------------
14a1377c 2020-05-10 20:07:44 prometheus5
1070cc2a 2020-05-12 18:23:24 prometheus5
c144f7cb 2020-05-13 18:10:02 prometheus5
77a81e3c 2020-05-19 18:00:44 prometheus5
69878268 2020-05-19 22:03:41 prometheus5
677f3a96 2020-05-20 18:10:02 prometheus5
d06af92e 2020-05-21 18:10:02 prometheus5
cd496767 2020-05-22 18:10:02 prometheus5
--------------------------------------------------
8 snapshots
--------------------------------------------------------------------------------
e[33m[Cleaning Repo...]e[0m
Applying Policy: keep the last 90 snapshots snapshots and all snapshots with tags [[archive]]
keep 8 snapshots:
ID Time Host Tags Reasons Paths
-----------------------------------------------------------------------------------------------
14a1377c 2020-05-10 20:07:44 prometheus5 last snapshot /media/servers/ad/shares
1070cc2a 2020-05-12 18:23:24 prometheus5 last snapshot /media/servers/ad/shares
c144f7cb 2020-05-13 18:10:02 prometheus5 last snapshot /media/servers/ad/shares
77a81e3c 2020-05-19 18:00:44 prometheus5 last snapshot /media/servers/ad/shares
69878268 2020-05-19 22:03:41 prometheus5 last snapshot /media/servers/ad/shares
677f3a96 2020-05-20 18:10:02 prometheus5 last snapshot /media/servers/ad/shares
d06af92e 2020-05-21 18:10:02 prometheus5 last snapshot /media/servers/ad/shares
cd496767 2020-05-22 18:10:02 prometheus5 last snapshot /media/servers/ad/shares
-----------------------------------------------------------------------------------------------
8 snapshots
counting files in repo
building new index for repo
[28:43] 100.00% 115425 / 115425 packs
repository contains 115425 packs (1477819 blobs) with 529.070 GiB
processed 1477819 blobs: 0 duplicate blobs, 0 B duplicate
load all snapshots
find data that is still in use for 8 snapshots
[1:37] 100.00% 8 / 8 snapshots
found 1477819 of 1477819 data blobs still in use, removing 0 blobs
will remove 0 invalid files
will delete 0 packs and rewrite 0 packs, this frees 0 B
counting files in repo
[28:45] 100.00% 115425 / 115425 packs
finding old index files
saved new indexes as [7ee7bfc4 a114968b 01fb7e65 4d1ecb8e 58fb4d21 23571cfe f5b5f955 b92bbb20 c3187948 2399ddb5 db826a4f 156b543f 3d216003 724c6763 3ce68359 2cfde00a 408234c4 0380883b d4b2f6cc bf587662 8e62791c 8bd9c0eb 994879c8 02798a88 9d1c4ac7 0183b1f2 8b1f370d 1e2181d2 8665b52d 50559a15 dcaa209c 725ce783 fd2410d3 87572d78 51a3e66d 1af21a5f 2859b243 9ef417e6 3647df3f]
remove 75 old index files
done
--------------------------------------------------------------------------------
e[33m[Checking for Errors in Repo...]e[0m
using temporary cache in /tmp/restic-check-cache-242871920
created new cache in /tmp/restic-check-cache-242871920
create exclusive lock for repository
load indexes
check all packs
check snapshots, trees and blobs
no errors were found
--------------------------------------------------------------------------------
e[33m[Done Cleaning; Next Cleanup and Check Will Be Done in 1 days...]e[0m
--------------------------------------------------------------------------------
Processing stats : [####----------------](20%)
Processing stats : [########------------](40%)
Processing stats : [############--------](60%)
Processing stats : [################----](80%)
Processing stats : [####################](100%)
Summarized Info Restore Size Deduplicated Size
--------------------------------------------------------------------------------
Latest Snapshot 750.325 GiB 519.384 GiB
All Snapshots 755.659 GiB 528.971 GiB
--------------------------------------------------------------------------------
e[33mEnd:e[0m Sat May 23 2020 06:42:38 AM
e[33mDuration:e[0m 12 hours, 32 minutes and 37 seconds
================================================================================
SCRIPT ENDED
================================================================================
The 11 hours backup duration seem to caused by something outside of restic: The âscanâ of the directory tree took over 3 hours whereas in an older log it completed in less than an hour. And the scan functionality is rather simple, it just traverses the filesystem counting the files and directories.
The code screenshot confirms that the last crash containing chunker.go:303 did not have my proposed if-modification. If the error occurs you should get a line starting with error: ReadFull returned .
You also might want to investigate the errors reported by during the backup run (and maybe anonymize the filepaths). All those files are not included in the backup.
Thanks for the reply, so deleted all the backup and redid it again been two days so far and 5 hours seems decent
Ill keep you posted to see what happens
uploaded intermediate index 28b9d181
Files: 570 new, 101 changed, 1506387 unmodified
Dirs: 0 new, 3 changed, 0 unmodified
Data Blobs: 1233 new
Tree Blobs: 4 new
Added to the repo: 934.497 MiB
processed 1507058 files, 764.353 GiB in 5:03:43
snapshot 1db4db10 saved
There are 6 exclusion rules...
--------------------------------------------------------------------------------
[Snapshots List...]
ID Time Host Tags
--------------------------------------------------
98711929 2020-06-13 15:30:19 prometheus4
1db4db10 2020-06-15 18:10:02 prometheus4
--------------------------------------------------
2 snapshots
--------------------------------------------------------------------------------
Next cleanup and check in 2 hours and 52 minutes...
--------------------------------------------------------------------------------
Summarized Info Restore Size Deduplicated Size
--------------------------------------------------------------------------------
Latest Snapshot 764.352 GiB 532.638 GiB
All Snapshots 1.493 TiB 533.561 GiB
--------------------------------------------------------------------------------
End: Mon Jun 15 2020 11:19:30 PM
Duration: 5 hours, 9 minutes and 29 seconds
================================================================================
SCRIPT ENDED
================================================================================
