Hi all; did some further testing with some real loads.
Leaving for folks later on for info on how progress meter works (particularly with my backend - Onedrive for Business via rclone).
Happy to do more testing if anyone has suggestions.
The tl;dr - there is a step in middle (data files processed) where the progress seems to get stuck (see details below on lead up and output). The whole process did eventually work without a hitch but did take a decent while (total - 40 minutes with the no-progress sections accounting for 2/3 of that…)
Got stuck at around ~400 or so - very fast 1 minute to 400 and then next 9 minutes to get to 402
Terminated via control c and the unlock and try again.
Next run fine:
[3:10] 100.00% 1429 / 1429 files deleted
collect data files for deletion and repacking
Had for a very long time:
0% 0 / 2428
and then suddenly:
data file fbbcd5de is not referenced in any index and not used -> will be removed.
data file ff442e9d is not referenced in any index and not used -> will be removed.
[4:46] 100.00% 2428 / 2428 data files processed
repacking data files
Ok fast up initally but great slowdown around 5 mins; go from 56 to 60 over next 7 minutes
[11:36] 74.07% 60 / 81 data files repacked
deleting obsolete data files…
Instant to 69 but then:
[0:37] 51.11% 69 / 135 files deleted
took another 8 minutes to get to 74 files deleted but then ran really fast and finished