Commit 89bad26
### What changes were proposed in this pull request?
Refactoring of the `DateTimeUtils.getEpoch()` function by avoiding decimal operations that are pretty expensive, and converting the final result to the decimal type at the end.
### Why are the changes needed?
The changes improve performance of the `getEpoch()` method at least up to **20 times**.
Before:
```
Invoke extract for timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
cast to timestamp 256 277 33 39.0 25.6 1.0X
EPOCH of timestamp 23455 23550 131 0.4 2345.5 0.0X
```
After:
```
Invoke extract for timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
cast to timestamp 255 294 34 39.2 25.5 1.0X
EPOCH of timestamp 1049 1054 9 9.5 104.9 0.2X
```
### Does this PR introduce any user-facing change?
No
### How was this patch tested?
By existing test from `DateExpressionSuite`.
Closes #25881 from MaxGekk/optimize-extract-epoch.
Authored-by: Maxim Gekk <max.gekk@gmail.com>
Signed-off-by: HyukjinKwon <gurwls223@apache.org>
1 parent 3be5741 commit 89bad26
File tree
2 files changed
+92
-91
lines changed- sql
- catalyst/src/main/scala/org/apache/spark/sql/catalyst/util
- core/benchmarks
2 files changed
+92
-91
lines changedLines changed: 4 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
847 | 847 | | |
848 | 848 | | |
849 | 849 | | |
850 | | - | |
851 | | - | |
852 | | - | |
| 850 | + | |
| 851 | + | |
| 852 | + | |
| 853 | + | |
853 | 854 | | |
854 | 855 | | |
855 | 856 | | |
| |||
0 commit comments