You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[SPARK-50870][SQL] Add the timezone when casting to timestamp in V2ScanRelationPushDown
### What changes were proposed in this pull request?
Add the timezone information to a cast expression when the destination type requires it.
### Why are the changes needed?
When current_timestamp() is materialized as a string, the timezone information is gone (e.g., 2024-12-27 10:26:27.684158) which prohibits further optimization rules from being applied to the affected data source.
For example,
```
Project [1735900357973433#10 AS current_timestamp()#6]
+- 'Project [cast(2025-01-03 10:32:37.973433#11 as timestamp) AS 1735900357973433#10]
+- RelationV2[2025-01-03 10:32:37.973433#11] xxx
```
-> This query fails to execute because the injected cast expression lacks the timezone information.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Existing tests.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closesapache#49549 from changgyoopark-db/SPARK-50870.
Authored-by: changgyoopark-db <changgyoo.park@databricks.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
0 commit comments