Prune: data not found in the index

Hello,
Since a few days, restic prune reports that some data is “not found in the index” (see complete logs below).

{<data/85dde23b>} not found in the index

I tried to rebuild-index, but it did not help.

"logs
$ restic rebuild-index
repository feb1903f opened (version 1)
loading indexes...
getting pack files to read...
rebuilding index
[0:13] 100.00%  77774 / 77774 packs processed
deleting obsolete index files
[0:00] 100.00%  33 / 33 files deleted
done
$ restic forget --prune --keep-within-hourly 1d --keep-within-daily 7d --keep-within-weekly 1m --keep-within-monthly 6m
repository feb1903f opened (version 1)
Applying Policy: keep hourly snapshots within 1d, daily snapshots within 7d, weekly snapshots within 1m, monthly snapshots within 6m
keep 35 snapshots:
ID        Time                 Host        Tags        Reasons            Paths
-----------------------------------------------------------------------------------------
522876e4  2023-11-30 23:00:05  ubuntu                  monthly within 6m  /mnt/data
                                                                          /var/spool/cron

1c4e5805  2023-12-31 23:00:06  ubuntu                  monthly within 6m  /mnt/data
                                                                          /var/spool/cron

b265353a  2024-01-31 23:00:05  ubuntu                  monthly within 6m  /mnt/data
                                                                          /var/spool/cron

070fa28a  2024-02-04 23:00:04  ubuntu                  weekly within 1m   /mnt/data
                                                                          /var/spool/cron

8194a361  2024-02-11 23:00:04  ubuntu                  weekly within 1m   /mnt/data
                                                                          /var/spool/cron

d1466829  2024-02-18 23:00:04  ubuntu                  weekly within 1m   /mnt/data
                                                                          /var/spool/cron

f7d07c5e  2024-02-23 23:00:04  ubuntu                  daily within 7d    /mnt/data
                                                                          /var/spool/cron

48205380  2024-02-24 23:00:05  ubuntu                  daily within 7d    /mnt/data
                                                                          /var/spool/cron

8ab197fb  2024-02-25 23:00:04  ubuntu                  daily within 7d    /mnt/data
                                                       weekly within 1m   /var/spool/cron

bf2b323e  2024-02-26 23:00:04  ubuntu                  daily within 7d    /mnt/data
                                                                          /var/spool/cron

8733434c  2024-02-27 23:00:05  ubuntu                  daily within 7d    /mnt/data
                                                                          /var/spool/cron

3456b340  2024-02-28 23:00:06  ubuntu                  daily within 7d    /mnt/data
                                                                          /var/spool/cron

5f406743  2024-02-29 12:00:04  ubuntu                  hourly within 1d   /mnt/data
                                                                          /var/spool/cron

bb05a4b7  2024-02-29 13:00:05  ubuntu                  hourly within 1d   /mnt/data
                                                                          /var/spool/cron

482339cf  2024-02-29 14:00:04  ubuntu                  hourly within 1d   /mnt/data
                                                                          /var/spool/cron

9264f64f  2024-02-29 15:00:05  ubuntu                  hourly within 1d   /mnt/data
                                                                          /var/spool/cron

1309048e  2024-02-29 16:00:04  ubuntu                  hourly within 1d   /mnt/data
                                                                          /var/spool/cron

804d089c  2024-02-29 17:00:05  ubuntu                  hourly within 1d   /mnt/data
                                                                          /var/spool/cron

4c3cc4e4  2024-02-29 18:00:05  ubuntu                  hourly within 1d   /mnt/data
                                                                          /var/spool/cron

5e0edb0f  2024-02-29 19:00:04  ubuntu                  hourly within 1d   /mnt/data
                                                                          /var/spool/cron

2d21a149  2024-02-29 20:00:04  ubuntu                  hourly within 1d   /mnt/data
                                                                          /var/spool/cron

b8f2bc4a  2024-02-29 21:59:09  ubuntu                  hourly within 1d   /mnt/data
                                                                          /var/spool/cron

6ebd3464  2024-02-29 23:00:06  ubuntu                  hourly within 1d   /mnt/data
                                                       daily within 7d    /var/spool/cron
                                                       monthly within 6m

a64f5428  2024-03-01 00:00:04  ubuntu                  hourly within 1d   /mnt/data
                                                                          /var/spool/cron

55450c53  2024-03-01 01:00:05  ubuntu                  hourly within 1d   /mnt/data
                                                                          /var/spool/cron

279fbbba  2024-03-01 02:00:05  ubuntu                  hourly within 1d   /mnt/data
                                                                          /var/spool/cron

101a4f35  2024-03-01 03:00:05  ubuntu                  hourly within 1d   /mnt/data
                                                                          /var/spool/cron

c3d3fd1d  2024-03-01 04:00:05  ubuntu                  hourly within 1d   /mnt/data
                                                                          /var/spool/cron

c1324e76  2024-03-01 05:00:05  ubuntu                  hourly within 1d   /mnt/data
                                                                          /var/spool/cron

5b2969b3  2024-03-01 06:00:04  ubuntu                  hourly within 1d   /mnt/data
                                                                          /var/spool/cron

198ca140  2024-03-01 07:00:04  ubuntu                  hourly within 1d   /mnt/data
                                                                          /var/spool/cron

a906bfea  2024-03-01 08:00:06  ubuntu                  hourly within 1d   /mnt/data
                                                                          /var/spool/cron

5947a661  2024-03-01 09:00:05  ubuntu                  hourly within 1d   /mnt/data
                                                                          /var/spool/cron

1381051d  2024-03-01 10:00:05  ubuntu                  hourly within 1d   /mnt/data
                                                                          /var/spool/cron

017586b1  2024-03-01 11:00:04  ubuntu                  hourly within 1d   /mnt/data
                                                       daily within 7d    /var/spool/cron
                                                       weekly within 1m
                                                       monthly within 6m
-----------------------------------------------------------------------------------------
35 snapshots

remove 1 snapshots:
ID        Time                 Host        Tags        Paths
----------------------------------------------------------------------
a5b3fd5e  2024-02-29 11:00:04  ubuntu                  /mnt/data
                                                       /var/spool/cron
----------------------------------------------------------------------
1 snapshots

[0:00] 100.00%  1 / 1 files deleted
1 snapshots have been removed, running prune
loading indexes...
loading all snapshots...
finding data that is still in use for 35 snapshots
[0:16] 100.00%  35 / 35 snapshots
searching used packs...
{<data/85dde23b>} not found in the index

Integrity check failed: Data seems to be missing.
Will not start prune to prevent (additional) data loss!
Please report this error (along with the output of the 'prune' run) at
https://github.com/restic/restic/issues/new/choose
Fatal: index is not complete

Any ideas? :slight_smile:

I solved my issue:

  • Upgrade to latest restic. I was on 0.15.2, upgraded to 0.16.4
  • Run restic repair index (same output as rebuild-index before)
  • Run restic repair snapshots
  • Notice that two snapshots had broken files.
  • restic forget the two snapshots that were repaired
  • restic prune now completes without issues.

Thanks for this great piece of software! :slight_smile: