time: add time/tzdata package and timetzdata tag to embed tzdata in program

[ianlancetaylor]

Go programs currently rely on the system to provide timezone information in the tzdata database, or rely on GOROOT being accessible at run time in order to read $GOROOT/lib/timezoneinfo.zip. This is not always true, particularly on Windows (#21881) but also in some other environments (#38013).

Therefore, it is desirable for Go program to have some mechanism to directly embed tzdata information into the program itself, so that they don't have to rely on system timezone information that may be absent.

I make the following assumptions, which I think are reasonable:

• adding the timezone information into a program should be optional, as it increases the size of the program by approximately 800K, and most programs do not need timezone information other than for the local system
• it should be possible to embed timezone information when building an arbitrary program
• it should be possible for a program to always embed timezone information without requiring any special build step
• the program should always prefer data from the system if available, as it is likely to be more up to date than that included in the program

Given those assumptions, I propose adding a new package timezone/tzdata. Importing this package (as import _ "time/tzdata") will cause timezone information to be embedded in the program. Also, building with the build tag timetzdata will force the package to be imported, and therefore will cause timezone information to be embedded in the program. The embedded timezone information will be used if and when no system information is available for the timezone.

There is a sample implementation of this approach at https://golang.org/cl/224588.

[gopherbot]

Change https://golang.org/cl/224588 mentions this issue: time/tzdata: new package

[leighmcculloch]

I think this idea of supporting it as a compiler flag and as a package is great.

However, I'm wondering about whether the operating system tzdata should always be preferred if present as it may make an application unpredictable in cases where only using embedded data is highly preferred.

Would it make sense to make the time/tzdata work like certificate pools and root CAs work? Any app can embed its own set of root CAs by creating a cert pool and adding that cert pool to the http package's default transport. The Go stdlib could still have the tzdata package, and still hook up an embedded tzdata, but I think the way it gets hooked up in the code-imports-a-package version could be less magical than an anonymous import. We could require code explicitly setting something like time.DefaultLoadTzinfo = tzdata.LoadTzinfo variable which would allow a developer to control exactly when or how the tzdata gets used and it communicates what's happening to a reader much clearer.

It would support both cases:

• Making it a backup source of tzdata (replace DefaultLoadTzinfo with a function that tries calling the default DefaultLoadTzinfo and if it it comes back with nothing, then calls tzdata's)
• Making it an authoritative source (replace DefaultLoadTzinfo with tzdata's function)

Some of this comment was originally posted at #21881 (comment).

[zikaeroh]

Given those assumptions, I propose adding a new package timezone/tzdata. Importing this package (as import _ "time/tzdata") will cause timezone information to be embedded in the program. Also, building with the build tag timetzdata will force the package to be imported, and therefore will cause timezone information to be embedded in the program.

I would support the latter (the build tag), but I'm not sure about offering the former (the _ imported package). I can foresee some time helper package that wants to "make sure" there's timezone information including this, and it ending up in people's dependency trees with no way to exclude it other than to pester the library author or fork.

The _ package is common to load dependencies like database drivers, but I think the binary size impact of those packages is lower than the 800K blob in comparison.

[glycerine]

I really like the proposal. A small caveat about that last assumption,

the program should always prefer data from the system if available, as it is likely to be more up to date than that included in the program

Like @leighmcculloch 's comment, I would prefer to give priority to the embedded data always if it is available, so that I don't have deal with platform specific heisenbugs due to data differences or if the user has messed up their system timezone files.

But either way, this is a 1000% improvement on having to hack the Go runtime after every release.

[mvdan]

I wonder if we should wait for more progress on #35950, given that one could think of tzdata as an opt-in asset file. Either way, I agree that a solution to this issue would be very useful.

[ianlancetaylor]

The kinds of problems that people describe with using system timezone data first are problems that people would see today, where we only use system timezone data. Have people actually seen, in practice, problems across different systems other than missing timezone information?

My concern with using embedded timezone data first is that it is guaranteed to be out of date within six months. While most programs are rebuilt within that time frame, there are certainly programs that are built and deployed once. Those programs may embed timezone information because they run in restricted environments, but they should still use updated timezone information as it becomes available. I have seen, in practice, problems with out of date timezone information, though it was with out of date timezone information on the system rather than in the program.

I don't particularly want to give programs a choice here. That leads to a more complicated API with benefits for few.

[ianlancetaylor]

@zikaeroh I think we can address that issue with documentation.

[glycerine]

It is difficult to make predictions. Especially about the future. ~ Danish proverb

I've reconsidered. I think it is fine as proposed.

[leighmcculloch]

My concern with using embedded timezone data first is that it is guaranteed to be out of date within six months.

That's a great point. 👏 I think this approach is good then for solving this problem.

Reflecting on when I would use this feature, I struggle to see myself using it. In most cases I make sure tzdata is installed and my operating system stays up-to-date as a part of general system health in the same way that root CAs need to be present. The very rare case I needed consistent tzdata led me to 4d63.com/tz but I no longer have that problem. The issue referenced in the PR description of how we get tzdata in Windows will be addressed some other way (#21881). I realize relying on the operating system is not the workflow everyone uses given #38013 and so for those use cases this seems good.

[alexbrainman]

@ianlancetaylor

Over the years, I never needed time zone information in my program. I believe majority of Go users are in the same situation as me.

If this proposal is accepted, I am concerned that all my programs will become 800K larger. Same for many other users. I think some package developers will just add import _ "time/tzdata" to one of their source files, and all executables that use that package will silently become 800K larger.

Go executables are large as they are - #6853 - and a lot of developers spend their time making executables smaller. Every 1 K counts. And here we are just wasting 800K of their efforts on something that users won't even use.

• adding the timezone information into a program should be optional, as it increases the size of the program by approximately 800K, and most programs do not need timezone information other than for the local system

How do you propose I will control if 800K is added to my executable or not?

The only reasonable solution I see here is to restrict import _ "time/tzdata" to main package only.

Thank you.

Alex

PS: Some runt. If this issue is to please users complaining about #21881, then someone should just spent some time and implement #21881 (comment) or #21881 (comment)

[leighmcculloch]

The only reasonable solution I see here is to restrict import _ "time/tzdata" to main package only.

This could be an argument to make it a build tag only and leave out the anonymous import.

someone should just spent some time and implement #21881 (comment)

Maybe we could wait for #21881 (comment) to be implemented, and then evaluate if the remaining use cases remaining warrant it?