Skip to content

feat: extract TimeType to int/decimal#4997

Open
YutaLin wants to merge 5 commits into
apache:mainfrom
YutaLin:4983_TimeType_to_Int_Decimal
Open

feat: extract TimeType to int/decimal#4997
YutaLin wants to merge 5 commits into
apache:mainfrom
YutaLin:4983_TimeType_to_Int_Decimal

Conversation

@YutaLin

@YutaLin YutaLin commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Closes #4983

Rationale for this change

Support time extraction (from TimeType to Int/Decimal)

SQL Spark expression StaticInvoke method Return type
extract(HOUR FROM t) HoursOfTime getHoursOfTime IntegerType
extract(MINUTE FROM t) MinutesOfTime getMinutesOfTime IntegerType
extract(SECOND FROM t) SecondsOfTime getSecondsOfTime IntegerType
extract(SECOND FROM t) (with fraction) SecondsOfTimeWithFraction getSecondsOfTimeWithFraction DecimalType

What changes are included in this PR?

  • Adds native Comet support for extracting hour, minute, and second from Spark TimeType values.
  • Adds serde handling for Spark’s DateTimeUtils StaticInvoke expressions used by TimeType extraction.
  • Implements fractional-second extraction as Decimal(8, 6) from Arrow Time64(Nanosecond) values.
  • Registers the new second_with_fraction native scalar function.
  • Adds Rust unit tests and Spark SQL coverage for hour, minute, second, extract(SECOND FROM ...), and date_part('SECOND', ...).

How are these changes tested?

  • Added Rust unit tests for extracting hour, minute, integer second, and fractional second from Time64(Nanosecond), including multiple precision values.
  • Added Spark SQL tests in time_extract.sql covering hour, minute, second, extract(SECOND FROM ...), and date_part('SECOND', ...).
  • Verified the SQL results against Spark while requiring native Comet execution.
  • Ran the SQL test suite with Spark 4.1.

@YutaLin

YutaLin commented Jul 21, 2026

Copy link
Copy Markdown
Contributor Author

Hi @parthchandra, could you help me review this? Thanks!

cc @andygrove

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement TimeType support - Time extraction (from TimeType to Int/Decimal)

1 participant