Which timezone does `forget` use?


#1

When running restic forget --keep-daily or --keep-weekly etc, restic apparently uses the natural calendar instead of relative time interval (which makes sense, I think), but which timezone is used?
The one where the forget command is run? The ones where the backups were made? Or simply UTC?

I’m thinking about this because I worry about unexpected snapshot/data loss due to timezone change. An (admittedly somewhat contrived) example:
Say I make daily snapshots. On odd numbered days, snapshots are made in the morning; whereas on even numbered days, snapshots are made in the evening. Running restic forget --keep-daily <large_number> will keep all these snapshots.
Now we change to a different timezone, then every two snapshots may land in the same calendar day in the new local time. If restic uses the local time where the forget command is run, the same command could cut the snapshots number in half!

I vaguely remember seeing related issues discussed on GitHub but can’t seem to find them.


#2

Hm, that’s an interesting issue you raise here, I haven’t thought about that at all! I think it would be valuable to investigate what restic does in this case. Please note that restic does not use the current date/time at all when checking --keep-daily, but rather the timestamps when the snapshots have been made. A snapshot’s timestamp always includes the timezone in which it was made.

I did a quick test and confirmed the following behavior:

  • For the relative conditions (e.g. --keep-within), the timezone is used to correctly compute the time between the snapshots
  • For the other conditions (e.g. --keep-daily), the “day” for a snapshot is computed while taking into account the timezone, so two snapshots made on 2018-11-03T01:38:41+01:00, and 2018-11-04T01:38:41+06:00 count as two different days, although they would fall onto the same day when converted to e.g. CET (+01:00).

So, in conclusion, restic does the right thing here (I think). But that was more or less by accident :wink:

What do you mean by “natural calendar”? The common Julian calendar?

For the record, this is the patch I used to investigate:

diff --git a/internal/restic/snapshot_policy_test.go b/internal/restic/snapshot_policy_test.go
index a9a80dd7..81196e1d 100644
--- a/internal/restic/snapshot_policy_test.go
+++ b/internal/restic/snapshot_policy_test.go
@@ -266,3 +266,48 @@ func TestApplyPolicy(t *testing.T) {
                })
        }
 }
+
+func TestApplyPolicyTimezones(t *testing.T) {
+       parse := func(s string) time.Time {
+               ts, err := time.Parse(time.RFC3339, s)
+               if err != nil {
+                       t.Fatalf("errors parsing timestamp %v: %v", s, err)
+               }
+               return ts
+       }
+
+       var snapshots = restic.Snapshots{
+               {Time: parse("2018-11-01T01:38:41+01:00")},
+               {Time: parse("2018-11-02T01:38:41+01:00")},
+               {Time: parse("2018-11-03T01:38:41+01:00")},
+               {Time: parse("2018-11-04T01:38:41+06:00")},
+       }
+
+       var tests = []restic.ExpirePolicy{
+               {Daily: 2},
+               {Within: parseDuration("2d")},
+       }
+
+       for _, p := range tests {
+               t.Run("", func(t *testing.T) {
+
+                       for _, s := range snapshots {
+                               t.Logf("snapshot: %v", s.Time.Local())
+                       }
+
+                       t.Logf("policy: %v", p)
+
+                       keep, remove, _ := restic.ApplyPolicy(snapshots, p)
+
+                       t.Logf("keep %v, remove %v", len(keep), len(remove))
+
+                       for _, s := range keep {
+                               t.Logf("keep %v", s.Time.Local())
+                       }
+
+                       for _, s := range remove {
+                               t.Logf("remove %v", s.Time.Local())
+                       }
+               })
+       }
+}

#3

Thanks for the test!

I guess this means that restic forget uses the timezone as specified in the snapshots then. I do think this is the most reasonable option. It provides stable behavior without resorting to UTC, which is less user-friendly.

By “natural” I only mean to distinguish calendar days vs. time interval relative to current time.


When trying to do my own tests (without writing Go code), I noticed that the --time flag in restic backup does not accept time zones, and the accepted (timezone-less) time is set to UTC instead of local time, which is probably not most users expect. I should probably open an issue on GitHub.

Edit: Just noticed that there’s already an issue for this: #2065