You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[SPARK-39961][SQL] DS V2 push-down translate Cast if the cast is safe
### What changes were proposed in this pull request?
Currently, DS V2 push-down translate `Cast` only if the ansi mode is true.
In fact, if the cast is safe(e.g. cast number to string, cast int to long), we can translate it too.
This PR will call `Cast.canUpCast` so as we can translate `Cast` to V2 `Cast` safely.
Note: The rule `SimplifyCasts` optimize some safe cast, e.g. cast int to long, so we may not see the `Cast`.
### Why are the changes needed?
Add the range for DS V2 push down `Cast`.
### Does this PR introduce _any_ user-facing change?
'Yes'.
`Cast` could be pushed down to data source in more cases.
### How was this patch tested?
Test cases updated.
Closes#37388 from beliefer/SPARK-39961.
Authored-by: Jiaan Geng <beliefer@163.com>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
0 commit comments