Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement date/time -> string formatting functions #9147

Closed
alamb opened this issue Feb 7, 2024 · 3 comments · Fixed by #9181
Closed

Implement date/time -> string formatting functions #9147

alamb opened this issue Feb 7, 2024 · 3 comments · Fixed by #9181
Labels
enhancement New feature or request

Comments

@alamb
Copy link
Contributor

alamb commented Feb 7, 2024

Is your feature request related to a problem or challenge?

Asked about in Discord

DataFusion doesn't have any time formatting functions, right? i.e. something like date_time.format("%d/%m/%Y")

The idea is to be able to format timestamps / dates back to different formats of strings

Describe the solution you'd like

Implement some sort of date/time/timestamp -> string formatting function that takes a user defined formatting string (probably chrono format) and returns a string

For example the postgres to_char function does this:
https://www.postgresql.org/docs/current/functions-formatting.html

Spark may have a better API (I haven't researched it)

arrow-rs already has the actual formatting code in ArrayFormatter:
https://docs.rs/arrow/latest/arrow/util/display/struct.ArrayFormatter.html

And specify by
https://docs.rs/arrow/latest/arrow/util/display/struct.FormatOptions.html

So this project is likely just to define a function, wire it in, and write some tests

Describe alternatives you've considered

No response

Additional context

No response

@alamb alamb added the enhancement New feature or request label Feb 7, 2024
@Omega359
Copy link
Contributor

Omega359 commented Feb 7, 2024

I'll work on this next if no one minds

@Omega359
Copy link
Contributor

Omega359 commented Feb 7, 2024

Spark api has a different name (date_format) but otherwise is very similar. I plan on having both names and have the spark one be an alias.

The pg to_char function also handles numbers which I think should be something we can either handle in this ticket or as a future improvement.

@alamb
Copy link
Contributor Author

alamb commented Feb 8, 2024

Thank you @Omega359

Omega359 added a commit to Omega359/arrow-datafusion that referenced this issue Feb 9, 2024
alamb added a commit that referenced this issue Feb 13, 2024
* initial to_char impl #9147

* fleshed out to_char impl, added tests and docs.

* adding sqllogictests

* adding time support, improved tests in datatime_expressions.rs, fixed scalar doc example

* Doc fix.

* Prettier

* Update datafusion/physical-expr/src/datetime_expressions.rs

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

* Updating the chrono dependency since Chrono 0.4.34 changed Display for the Duration type (which is now an alias for TimeDelta) and this broke the timestamp tests.

* Updates and fixes based on PR feedback.

* Add escape to rustdoc.

* Fixed example

* Handled null formats better - now any null formats will result in a default format being used.

---------

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants