Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expand time units for ttl config option #33686

Open
SpencerTorres opened this issue Jun 20, 2024 · 3 comments
Open

Expand time units for ttl config option #33686

SpencerTorres opened this issue Jun 20, 2024 · 3 comments
Labels

Comments

@SpencerTorres
Copy link
Member

Component(s)

exporter/clickhouse

Describe the issue you're reporting

When I saw we had two config options, ttl_days and ttl, it made sense to pick one (as we did recently by removing ttl_days in #33648)

Unfortunately when I approved this, I didn't realize that the largest unit of time for time.Duration is hours (h)

For telemetry use cases (mainly logging), data is kept for much longer than most people measure in hours. For example, I know a day is 24h, but what if I want to keep my log data for 6 months? While configuring my exporter I thought I could simply do 180d, but this of course fails to parse. It's not that doing the calculation to write 4320h is hard, it's just incredibly unintuitive. It's an easy mistake to make to type d there, especially for those who don't know Go and the time package.

The majority of units in time.ParseDuration are too small to be usable for a Database TTL config.

I believe it would be convenient to expand this to at least include days (d), but if we were to do this, where would be the best place?

Here are the different options, in order of least complexity:

  1. re-add ttl_days. We can still mark it as deprecated, but it is very convenient.
  2. In clickhouseexporter: we can change ttl from time.Duration to string and parse it in the local config parser function. We would add some logic for when the time contains d, but otherwise fallback to to time.ParseDuration.
  3. In OTel confmap: We can modify the hook here, so that all other components and configs can benefit from the change. We would have to write our own hook with the same change as option 1.
  4. In go-viper: This is the top-most parser we could modify without modifying Go itself. time.Duration does not have Unmarshal* functions declared, so this module adds a hook to include it. We could add a new function called StringToTimeDurationExtendedHookFunc to expand the current behavior, and then use it in OTel confmap.

This would be a non-breaking change since it doesn't affect any of the existing units. This change could benefit all OTel configs, not just ClickHouse exporter. Let me know what you think.

Thanks!

Note in case anyone points it out: This convenience is for users on Earth, and the conventional understanding that a day is 24 hours, rounded for convenience.

Copy link
Contributor

Pinging code owners for exporter/clickhouse: @hanjm @dmitryax @Frapschen @SpencerTorres. See Adding Labels via Comments if you do not have permissions to add labels yourself.

@crobert-1
Copy link
Member

I don't have super strong opinions here, just sharing some thoughts.

It looks like the reason d isn't supported by the time library is exactly what you're pointing out in your note at the end, as explained here.

Another option for users is to simply add a comment when setting the config option. Something like: ttl: 4320h # 180 days It seems like a user could set it, and another user could simply read this line and understand what's going on. I don't think it's that confusing, all in all.

I'll defer to code owners though.

@crobert-1 crobert-1 added enhancement New feature or request and removed needs triage New item requiring triage labels Jul 15, 2024
Copy link
Contributor

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot added the Stale label Sep 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants