Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

date_trunc always returns Timestamp(Nanosecond, None) which might truncate ranges #6653

Closed
Tracked by #3148
alamb opened this issue Jun 12, 2023 · 2 comments · Fixed by #6654
Closed
Tracked by #3148

date_trunc always returns Timestamp(Nanosecond, None) which might truncate ranges #6653

alamb opened this issue Jun 12, 2023 · 2 comments · Fixed by #6654
Labels
enhancement New feature or request

Comments

@alamb
Copy link
Contributor

alamb commented Jun 12, 2023

Is your feature request related to a problem or challenge?

Prior to #6632 and #6638 the date_trunc function was inconsistent between reported types and actual types for arrays/scalars

Now it is consistent in the sense that

date_trunc(Timestamp(Second, ..)) --> Timestamp(Nanosecond, None)
date_trunc(Timestamp(Millisecond, ..)) --> Timestamp(Nanosecond, None)
date_trunc(Timestamp(Microsecond ..)) --> Timestamp(Nanosecond, None)
date_trunc(Timestamp(Nanosecond ..)) --> Timestamp(Nanosecond, None)

However, by using Timestamp(Nanosecond, None) everywhere will truncate the range over which date_trunc will work -- specifically if there is some value that can be represented using Timestamp(Second) that doesn't fit into a Timestamp(Nanosecond) such as a value before 1677-09-21 00:12:43.145224193 or after 2262-04-11 23:47:16.854775807 date_trunc will error.

Describe the solution you'd like

We could potentially change the function to keep the same TimeUnit of its input:

Namely

date_trunc(Timestamp(Second, ..)) --> Timestamp(Second, None)
date_trunc(Timestamp(Millisecond, ..)) --> Timestamp(Millisecond, None)
date_trunc(Timestamp(Microsecond ..)) --> Timestamp(Microsecond, None)
date_trunc(Timestamp(Nanosecond ..)) --> Timestamp(Nanosecond, None)

Describe alternatives you've considered

We could also just leave the current behavior

Additional context

See comments #6632 (comment)

@viirya
Copy link
Member

viirya commented Jun 12, 2023

date_trunc(Timestamp(Second, ..)) --> Timestamp(Second, None)

If granularity parameter is smaller than second, e.g. millisecond, or nanosecond, what is the expected behavior? Returns original input, or throws an error?

@alamb
Copy link
Contributor Author

alamb commented Jun 12, 2023

If granularity parameter is smaller than second, e.g. millisecond, or nanosecond, what is the expected behavior? Returns original input, or throws an error?

I would think returning the original would make the most sense and still be accurate. But it definitely wouldn't be very useful 🤔

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants