Pruning failure

Restic cannot delete packs after pruning. The half-dozen or so packs slotted for removal also have corresponding ._ files. Yet even when these are removed pruning still fails and they are recreated. The repository is located on a NAS.

Restic version:

restic 0.17.3 compiled with go1.23.3 on darwin/arm64

Command run (no env variables, pw entered at prompt)

restic --verbose=2 prune -r documents

The following output shows 5 things

  1. The output of the prune command
  2. The listing of one of the failed packs’ parent directory
  3. Removal of the ._ file
  4. The output of the prune command again
  5. The listing again with the newly created ._ file. Different timestamp.
❯ restic --verbose=2 prune -r documents
enter password for repository:
repository 0f1f3bda opened (version 2, compression level auto)
loading indexes...
[0:00] 100.00%  5 / 5 index files loaded
loading all snapshots...
finding data that is still in use for 9 snapshots
[0:00] 100.00%  9 / 9 snapshots
searching used packs...
collecting packs for deletion and repacking
will remove pack 3e206276 as it is unused and not indexed
will remove pack 52e0accc as it is unused and not indexed
will remove pack 6086ef74 as it is unused and not indexed
will remove pack 6371b800 as it is unused and not indexed
will remove pack 8ea9d200 as it is unused and not indexed
will remove pack a81ee3df as it is unused and not indexed
will remove pack cd3dcc47 as it is unused and not indexed
will remove pack de8738cf as it is unused and not indexed
[0:52] 100.00%  1628 / 1628 packs processed

used:             188836 blobs / 26.272 GiB
unused:                0 blobs / 0 B
unreferenced:                    139.724 MiB
total:            188836 blobs / 26.408 GiB
unused size: 0.00% of total size

to repack:             0 blobs / 0 B
this removes:          0 blobs / 0 B
to delete:             0 blobs / 139.724 MiB
total prune:           0 blobs / 139.724 MiB
remaining:        188836 blobs / 26.272 GiB
unused size after prune: 0 B (0.00% of remaining size)

totally used packs:       1628
partly used packs:           0
unused packs:                0

to keep:            1628 packs
to repack:             0 packs
to delete:             0 packs
to delete:             8 unreferenced packs

deleting unreferenced packs
Remove(<data/8ea9d20026>) failed: remove documents/data/8e/8ea9d20026d18077ae13116d949cb03bbca8f055a4b9ae6915a7a0e7d776ab70: operation not permitted
unable to remove data/8ea9d20026d18077ae13116d949cb03bbca8f055a4b9ae6915a7a0e7d776ab70 from the repository
removed data/8ea9d20026d18077ae13116d949cb03bbca8f055a4b9ae6915a7a0e7d776ab70
Remove(<data/6371b8007f>) failed: remove documents/data/63/6371b8007f8a6aaa673e7465eb8b671dbff10a32964256ad997d98c9235b2ada: operation not permitted
unable to remove data/6371b8007f8a6aaa673e7465eb8b671dbff10a32964256ad997d98c9235b2ada from the repository
removed data/6371b8007f8a6aaa673e7465eb8b671dbff10a32964256ad997d98c9235b2ada
Remove(<data/a81ee3dfa5>) failed: remove documents/data/a8/a81ee3dfa5fb887fb5daa2485b180f2804698ed28323633c467878daab5d3af2: operation not permitted
unable to remove data/a81ee3dfa5fb887fb5daa2485b180f2804698ed28323633c467878daab5d3af2 from the repository
removed data/a81ee3dfa5fb887fb5daa2485b180f2804698ed28323633c467878daab5d3af2
Remove(<data/cd3dcc47f4>) failed: remove documents/data/cd/cd3dcc47f4859d9af655e6cdcf41593ef63dfeb5b64efac7f79eaf8d29ffc7b9: operation not permitted
unable to remove data/cd3dcc47f4859d9af655e6cdcf41593ef63dfeb5b64efac7f79eaf8d29ffc7b9 from the repository
removed data/cd3dcc47f4859d9af655e6cdcf41593ef63dfeb5b64efac7f79eaf8d29ffc7b9
Remove(<data/de8738cf8c>) failed: remove documents/data/de/de8738cf8c64650ba2b4c6968a80f3c65ca78e51c93371327a186b82032e66c9: operation not permitted
unable to remove data/de8738cf8c64650ba2b4c6968a80f3c65ca78e51c93371327a186b82032e66c9 from the repository
removed data/de8738cf8c64650ba2b4c6968a80f3c65ca78e51c93371327a186b82032e66c9
Remove(<data/3e206276aa>) failed: remove documents/data/3e/3e206276aa53f24163781d6a460fab0dc561ccb9f9ac8c249cd2b04d15e62707: operation not permitted
unable to remove data/3e206276aa53f24163781d6a460fab0dc561ccb9f9ac8c249cd2b04d15e62707 from the repository
removed data/3e206276aa53f24163781d6a460fab0dc561ccb9f9ac8c249cd2b04d15e62707
Remove(<data/52e0acccb9>) failed: remove documents/data/52/52e0acccb92d21e5468495760cf75050be4e6e457e984811478adc97c776ec07: operation not permitted
unable to remove data/52e0acccb92d21e5468495760cf75050be4e6e457e984811478adc97c776ec07 from the repository
removed data/52e0acccb92d21e5468495760cf75050be4e6e457e984811478adc97c776ec07
Remove(<data/6086ef745f>) failed: remove documents/data/60/6086ef745f804c1aa51e19070bf699b6cab46762f79f17761e32b0be11950adf: operation not permitted
unable to remove data/6086ef745f804c1aa51e19070bf699b6cab46762f79f17761e32b0be11950adf from the repository
removed data/6086ef745f804c1aa51e19070bf699b6cab46762f79f17761e32b0be11950adf
[0:00] 100.00%  8 / 8 files deleted
done
❯ ls -la documents/data/de
total 199840
drwx------  1 john  staff     16384 Jan 26 13:53 .
drwx------  1 john  staff     16384 Jan 14 23:44 ..
-rwx------  1 john  staff      4096 Jan 26 13:53 ._de8738cf8c64650ba2b4c6968a80f3c65ca78e51c93371327a186b82032e66c9
-rwx------  1 john  staff  17181351 Sep  1 14:41 de04089e78270f78826fa894dc252560e93c9ccbde92938cecdeca62108f68e3
-rwx------  1 john  staff  16834480 Mar 13  2024 de4567af288190fbc6b9f79954a328e97a02993aa9fd8afdb7df58d3a515a245
-rwx------  1 john  staff  16961659 Sep  1 14:42 de47cfb0d2a54aa2e5ca4c3d59ffc5980325e15a0542877e3750da5fc2305d05
-rwx------@ 1 john  staff  17393474 Apr  6  2024 de8738cf8c64650ba2b4c6968a80f3c65ca78e51c93371327a186b82032e66c9
-rwx------  1 john  staff  16888073 Mar 13  2024 dec132ce7d4788ae2121ec17f72174b2eecf41171682950ea5f094cb281137d6
-rwx------  1 john  staff  17010283 Sep  1 14:41 ded0c5dd6498960ff468a26f27c2d8542cfdd364e69f75e6963697e5b8b825e6
❯ rm documents/data/de/._de8738cf8c64650ba2b4c6968a80f3c65ca78e51c93371327a186b82032e66c9
❯ restic --verbose=2 prune -r documents
enter password for repository:
repository 0f1f3bda opened (version 2, compression level auto)
loading indexes...
[0:00] 100.00%  5 / 5 index files loaded
loading all snapshots...
finding data that is still in use for 9 snapshots
[0:00] 100.00%  9 / 9 snapshots
searching used packs...
collecting packs for deletion and repacking
will remove pack 3e206276 as it is unused and not indexed
will remove pack 52e0accc as it is unused and not indexed
will remove pack 6086ef74 as it is unused and not indexed
will remove pack 6371b800 as it is unused and not indexed
will remove pack 8ea9d200 as it is unused and not indexed
will remove pack a81ee3df as it is unused and not indexed
will remove pack cd3dcc47 as it is unused and not indexed
will remove pack de8738cf as it is unused and not indexed
[0:55] 100.00%  1628 / 1628 packs processed

used:             188836 blobs / 26.272 GiB
unused:                0 blobs / 0 B
unreferenced:                    139.724 MiB
total:            188836 blobs / 26.408 GiB
unused size: 0.00% of total size

to repack:             0 blobs / 0 B
this removes:          0 blobs / 0 B
to delete:             0 blobs / 139.724 MiB
total prune:           0 blobs / 139.724 MiB
remaining:        188836 blobs / 26.272 GiB
unused size after prune: 0 B (0.00% of remaining size)

totally used packs:       1628
partly used packs:           0
unused packs:                0

to keep:            1628 packs
to repack:             0 packs
to delete:             0 packs
to delete:             8 unreferenced packs

deleting unreferenced packs
Remove(<data/3e206276aa>) failed: remove documents/data/3e/3e206276aa53f24163781d6a460fab0dc561ccb9f9ac8c249cd2b04d15e62707: operation not permitted
unable to remove data/3e206276aa53f24163781d6a460fab0dc561ccb9f9ac8c249cd2b04d15e62707 from the repository
removed data/3e206276aa53f24163781d6a460fab0dc561ccb9f9ac8c249cd2b04d15e62707
Remove(<data/de8738cf8c>) failed: remove documents/data/de/de8738cf8c64650ba2b4c6968a80f3c65ca78e51c93371327a186b82032e66c9: operation not permitted
unable to remove data/de8738cf8c64650ba2b4c6968a80f3c65ca78e51c93371327a186b82032e66c9 from the repository
removed data/de8738cf8c64650ba2b4c6968a80f3c65ca78e51c93371327a186b82032e66c9
Remove(<data/52e0acccb9>) failed: remove documents/data/52/52e0acccb92d21e5468495760cf75050be4e6e457e984811478adc97c776ec07: operation not permitted
unable to remove data/52e0acccb92d21e5468495760cf75050be4e6e457e984811478adc97c776ec07 from the repository
removed data/52e0acccb92d21e5468495760cf75050be4e6e457e984811478adc97c776ec07
Remove(<data/6086ef745f>) failed: remove documents/data/60/6086ef745f804c1aa51e19070bf699b6cab46762f79f17761e32b0be11950adf: operation not permitted
unable to remove data/6086ef745f804c1aa51e19070bf699b6cab46762f79f17761e32b0be11950adf from the repository
removed data/6086ef745f804c1aa51e19070bf699b6cab46762f79f17761e32b0be11950adf
Remove(<data/6371b8007f>) failed: remove documents/data/63/6371b8007f8a6aaa673e7465eb8b671dbff10a32964256ad997d98c9235b2ada: operation not permitted
unable to remove data/6371b8007f8a6aaa673e7465eb8b671dbff10a32964256ad997d98c9235b2ada from the repository
removed data/6371b8007f8a6aaa673e7465eb8b671dbff10a32964256ad997d98c9235b2ada
Remove(<data/8ea9d20026>) failed: remove documents/data/8e/8ea9d20026d18077ae13116d949cb03bbca8f055a4b9ae6915a7a0e7d776ab70: operation not permitted
unable to remove data/8ea9d20026d18077ae13116d949cb03bbca8f055a4b9ae6915a7a0e7d776ab70 from the repository
removed data/8ea9d20026d18077ae13116d949cb03bbca8f055a4b9ae6915a7a0e7d776ab70
Remove(<data/a81ee3dfa5>) failed: remove documents/data/a8/a81ee3dfa5fb887fb5daa2485b180f2804698ed28323633c467878daab5d3af2: operation not permitted
unable to remove data/a81ee3dfa5fb887fb5daa2485b180f2804698ed28323633c467878daab5d3af2 from the repository
removed data/a81ee3dfa5fb887fb5daa2485b180f2804698ed28323633c467878daab5d3af2
Remove(<data/cd3dcc47f4>) failed: remove documents/data/cd/cd3dcc47f4859d9af655e6cdcf41593ef63dfeb5b64efac7f79eaf8d29ffc7b9: operation not permitted
unable to remove data/cd3dcc47f4859d9af655e6cdcf41593ef63dfeb5b64efac7f79eaf8d29ffc7b9 from the repository
removed data/cd3dcc47f4859d9af655e6cdcf41593ef63dfeb5b64efac7f79eaf8d29ffc7b9
[0:00] 100.00%  8 / 8 files deleted
done
❯ ls -la documents/data/de
total 199840
drwx------  1 john  staff     16384 Jan 26 14:32 .
drwx------  1 john  staff     16384 Jan 14 23:44 ..
-rwx------  1 john  staff      4096 Jan 26 14:33 ._de8738cf8c64650ba2b4c6968a80f3c65ca78e51c93371327a186b82032e66c9
-rwx------  1 john  staff  17181351 Sep  1 14:41 de04089e78270f78826fa894dc252560e93c9ccbde92938cecdeca62108f68e3
-rwx------  1 john  staff  16834480 Mar 13  2024 de4567af288190fbc6b9f79954a328e97a02993aa9fd8afdb7df58d3a515a245
-rwx------  1 john  staff  16961659 Sep  1 14:42 de47cfb0d2a54aa2e5ca4c3d59ffc5980325e15a0542877e3750da5fc2305d05
-rwx------@ 1 john  staff  17393474 Apr  6  2024 de8738cf8c64650ba2b4c6968a80f3c65ca78e51c93371327a186b82032e66c9
-rwx------  1 john  staff  16888073 Mar 13  2024 dec132ce7d4788ae2121ec17f72174b2eecf41171682950ea5f094cb281137d6
-rwx------  1 john  staff  17010283 Sep  1 14:41 ded0c5dd6498960ff468a26f27c2d8542cfdd364e69f75e6963697e5b8b825e6

What is the URL to this restic repository? How is it being accessed?

Files beginning with ._ are usually created by macOS. Presumably you have some macOS poking around in that filesystem, I guess you have it mounted on a Mac or something.

What NAS is it? Would it be possible to either have it serve an S3 compatible object storage, or run Minio or restic’s rest-server on it?

Regarding the “operation not permitted” messages, that’s something to consider once there is more info.

1 Like