Proper handling of timezone data

[iointerrupt]

Issue: Many running trackers do not properly handle dates and times, especially when accounting for timezones. If for example, I take a vacation to Europe and run in London at 8AM (GMT = UTC), and return home to new york (EST = UTC-5), it would show my london run time was at 3AM. Furthermore, if I run in new york at 8AM during eastern standing time (UTC-5), when the clocks springs ahead for daylight savings time (DST = UTC-4), it would be good to see my running time still as 8AM instead of 9AM. Keep in mind that the time is correctly stored already because its in UTC. Only my timezone offset has changed.

Solution: Proposing adding an additional column in tracks table for managing timezone offset of the current workout (tzoffset). The timezone can be retrieved from phone. The date and time is already stored properly using UTC regardless of the timzone because thats how GPS units timestamp the data. Storing the timezone offset is also a good idea to prevent having to do daylight savings calculations which could get messy if daylight savings times are ever done away with. For example, when I run in New York today, the tzoffset column can be stored as '-400'. When the time is shown (starttime column), you could offset with offset hours+minutes. I included the -400 instead of -4 because offsets can can also be in 30 minutes increments as is the case with Newfoundland in Canada. So an offset of 4 hours and 30 minutes can be '430' (reference: https://en.wikipedia.org/wiki/List_of_UTC_time_offsets). For historical workouts, if tzoffset column is NULL, consider it the current timezone. This will apply to all activities going forward.

[dennisguse]

Just out of my head (without checking the code).
We store the start and end of a track (as milliseconds since epoch; I guess without timezone as it uses System.currentTimeMillis();) and we store the UTC time from the GPS (if provided other we use System.currentTimeMillis();).

So, storing the timezone would be helpful on a per track basis.
See TracksColumns and it would be cool to make starttime and stoptime actually also UTC.

Another thing: we need to figure out, how to import/export this kind of data to GPX/KML.

@iointerrupt Feel free to dive into this topic!
It is probably an edge case, but also nice to implement this.

[iointerrupt]

The start time and end times are already stored based on UTC so we do not need to worry about that. The timezone offset of the device be determined in seconds from TimeZone.getDefault().getOffset(System.currentTimeMillis()) / 1000;

This is the value that could be stored in the tzOffset column on a per track basis.

When you display the track time on screen, apply this offset to the starttime and endtime and show that and dont rely on the current timezone of the system.

The GPX specification adheres to the ISO 8601 date/time format which supports the offset. Instead of exporting the time as "2020-06-29T18:34:47Z" if the tzOffset column is not NULL, you would change the string to something like "2020-06-29T18:34:47-05:00" or "2020-06-29T18:34:47+05:30". Below is code to convert the offset to a string format (e.g.: -18000 -> -05:00):

public static String getSystemTZOffsetStr() {
    int x = TimeZone.getDefault().getOffset(System.currentTimeMillis()) / 1000;
    return (x >= 0 ? "+" : "-") + String.format("%02d:%02d", Math.abs(x / 3600), Math.abs((x / 60) % 60));
}

Reference to ISO 8601: https://en.wikipedia.org/wiki/ISO_8601

[dennisguse]

Works for me.
I would store prefer storing the timezone's name (e.g., UTC) rather than the numerical offset to also include weird things like day light saving times.
So, why not extending Track/ TracksColumns with a timezone-column that contains this info?
We can just set the default value to UTC as this is current behavior, right?

Accounting for it in export / import should also be doable :)
I can recommend using ZonedDateTime should do this out of the box :)

PS: And KML also uses ISO 8601.

[iointerrupt]

I would advise against using the three-letter codes. I've read articles that storing the 3 digit timezone code is problematic because they have lead to confusions. For example CST = China Standard Time, but also CST = Central Standard Time (applies to north america). The approach going forward, and established in many date/time libraries has been to use the "Country/Region" code such as "[America/New_York]". But this will not account for the additional daylight savings offset which is observed by many European and North American countries. And it is exactly for this oddity it makes sense to store it via numerical offset.

If you do end up storing the region code or region name then you become unaware of the timeshift. A perfect example would be exporting and importing a GPX: Suppose for example I do a run today (June 30, 2020) in New York at 5:00pm local time (UTC Time is 21:00) The reporting time time today will be starting at: 5:00pm while the SAME run after November 1, 2020, it will show the start time as June 30, 2020 6:00pm.

Now, if I was to export the the GPX along with the offset using "2020-06-30T21:00:00-04:00" format, other systems whether its Strava, OpenTracksApp, whichever, stores the date/time as UTC (2020-06-30T21:00:00) and figures out how it would apply the offset (-04:00).

The correct solution would have been to store the time zone name (e.g.: [Europe/Paris] along with an additional column to offset whether its daylight savings or not. This is my understanding of how the ZonedDateTime api works: "A ZonedDateTime holds state equivalent to three separate objects, a LocalDateTime, a ZoneId and the resolved ZoneOffset.". I think this introduces far too much complexity.

EDIT: Additionally, The default value as current behavior, would be UTC as you mentioned. For GPX/KML exports, this would mean appending a 'Z' after the timestamp (e:g: 2020-06-30T21:00:00Z).

[iointerrupt]

I think instead of using ZonedDateTime, here you can utilize the OffsetDateTime class (https://docs.oracle.com/javase/10/docs/api/java/time/OffsetDateTime.html). This would allow the time to not be manipulated by the current local date/time zone rules but rather based fully on the offset.

[dennisguse]

@io-inter-upt Would you be able to work on this yourself?
And if yes, how can I help you?

[dennisguse]

We replaced the internal time representation (long in ms since epoch aka 1970) with Instant and Duration.
This should simplify the implementation of this functionality.

602f83f

[dennisguse]

And it get's slightly more complicated; just storing the offset for the Track would be possible, but would introduce some hacks.
Both KML and GPX export timestamps per location (in our case TrackPoint) with a timestamp: so on export would need to write the proper timezone everywhere.
And import compute back to the one timezone.

There are two edge cases:

• what happens if a user changes timezone (better time offset) while recording (e.g., driving from Spain to Portugal or recording while daylight saving changes; somebody might make a multi-day trip)?
• what happens if we have a track that contains multiple timezones (probably similar reasons)?

Another approach would be to store the time offset for each TrackPoint.
(and as starttime and stoptime is de-normalized for the Tracks: there as well).

@pstorch @rgmf Do you have an opinion about this?
PS/ I don't really need this feature, but I like the general idea of having this, because it is possible 😎

[pstorch]

Tricky problem. I try to avoid 😉

Oldie but goldie: https://youtu.be/-5wpm-gesOY

[dennisguse]

t9@pstorch Awesome!!!!!
That's what exactly what I needed - long story short: we should only store the zone offset once (aka in the Track).
And we will use this everytime, we render an Instant for the user (i.e., UI + export); during import, we just get the first one and normalize the other one's accordingly.

All internal handling will stay at epoch.

One night sleep + a video can change it all :D

[rgmf]

@pstorch great video 👍

[dennisguse]

* [x] add offset column to Track table
* [x] add field to Track
* [ ] add check in ExportImportTest
* [ ] use offset for UI renderings: StatisticsRecordedFragment
* [ ] use offset for naming tracks (default name option)
* [ ] use offset for UI renderings: TrackListActivity
* [ ] use offset for UI renderings: MarkerListActivity

Moved to PR: #1042

[dennisguse]

#1042 (comment)

Needs some more work for the UI:

• use offset for UI renderings: StatisticsRecordedFragment
• use offset for naming tracks (default name option)
• use offset for UI renderings: TrackListActivity
• use offset for UI renderings: MarkerListActivity
• filtering UI

[rgmf]

@dennisguse Shall we move android.text.format.DateUtils to DateTimeFormatter? If so, we can avoid to propagate Context object to StringUtils.

For example:

Move this:

public static String formatDateTime(Context context, Instant time) {
        return DateUtils.formatDateTime(context, time.toEpochMilli(), DateUtils.FORMAT_SHOW_DATE | DateUtils.FORMAT_NUMERIC_DATE)
                + " " + DateUtils.formatDateTime(context, time.toEpochMilli(), DateUtils.FORMAT_SHOW_TIME);
    }

To:

public static String formatDateTime(OffsetDateTime odt) {
        return odt.format(DateTimeFormatter.ofLocalizedDateTime(FormatStyle.MEDIUM).withZone(odt.toZonedDateTime().getZone()));
    }

[dennisguse]

@rgmf Looks good!

[dennisguse]

PS: technically, we don't have the timezone - it is just the offset that was present at the time of recording.
So, we cannot really derive the timezone from anymore.