Commit ea525fe
[SPARK-31597][SQL] extracting day from intervals should be interval.days + days in interval.microsecond
### What changes were proposed in this pull request?
With suggestion from cloud-fan #28222 (comment)
I Checked with both Presto and PostgresSQL, one is implemented intervals with ANSI style year-month/day-time, and the other is mixed and Non-ANSI. They both add the exceeded days in interval time part to the total days of the operation which extracts day from interval values.
```sql
presto> SELECT EXTRACT(DAY FROM (cast('2020-01-15 00:00:00' as timestamp) - cast('2020-01-01 00:00:00' as timestamp)));
_col0
-------
14
(1 row)
Query 20200428_135239_00000_ahn7x, FINISHED, 1 node
Splits: 17 total, 17 done (100.00%)
0:01 [0 rows, 0B] [0 rows/s, 0B/s]
presto> SELECT EXTRACT(DAY FROM (cast('2020-01-15 00:00:00' as timestamp) - cast('2020-01-01 00:00:01' as timestamp)));
_col0
-------
13
(1 row)
Query 20200428_135246_00001_ahn7x, FINISHED, 1 node
Splits: 17 total, 17 done (100.00%)
0:00 [0 rows, 0B] [0 rows/s, 0B/s]
presto>
```
```sql
postgres=# SELECT EXTRACT(DAY FROM (cast('2020-01-15 00:00:00' as timestamp) - cast('2020-01-01 00:00:00' as timestamp)));
date_part
-----------
14
(1 row)
postgres=# SELECT EXTRACT(DAY FROM (cast('2020-01-15 00:00:00' as timestamp) - cast('2020-01-01 00:00:01' as timestamp)));
date_part
-----------
13
```
```
spark-sql> SELECT EXTRACT(DAY FROM (cast('2020-01-15 00:00:00' as timestamp) - cast('2020-01-01 00:00:01' as timestamp)));
0
spark-sql> SELECT EXTRACT(DAY FROM (cast('2020-01-15 00:00:00' as timestamp) - cast('2020-01-01 00:00:00' as timestamp)));
0
```
In ANSI standard, the day is exact 24 hours, so we don't need to worry about the conceptual day for interval extraction. The meaning of the conceptual day only takes effect when we add it to a zoned timestamp value.
### Why are the changes needed?
Both satisfy the ANSI standard and common use cases in modern SQL platforms
### Does this PR introduce any user-facing change?
No, it new in 3.0
### How was this patch tested?
add more uts
Closes #28396 from yaooqinn/SPARK-31597.
Authored-by: Kent Yao <yaooqinn@hotmail.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>1 parent 6bc8d84 commit ea525fe
File tree
3 files changed
+22
-19
lines changed- sql
- catalyst/src
- main/scala/org/apache/spark/sql/catalyst/util
- test/scala/org/apache/spark/sql/catalyst/expressions
- core/src/test/resources/sql-tests/results
3 files changed
+22
-19
lines changedLines changed: 3 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
20 | | - | |
21 | 20 | | |
22 | 21 | | |
23 | 22 | | |
| |||
55 | 54 | | |
56 | 55 | | |
57 | 56 | | |
58 | | - | |
| 57 | + | |
| 58 | + | |
59 | 59 | | |
60 | 60 | | |
61 | 61 | | |
62 | | - | |
| 62 | + | |
63 | 63 | | |
64 | 64 | | |
65 | 65 | | |
| |||
Lines changed: 3 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
68 | 68 | | |
69 | 69 | | |
70 | 70 | | |
| 71 | + | |
71 | 72 | | |
72 | 73 | | |
73 | 74 | | |
| |||
81 | 82 | | |
82 | 83 | | |
83 | 84 | | |
| 85 | + | |
| 86 | + | |
84 | 87 | | |
85 | 88 | | |
86 | 89 | | |
| |||
Lines changed: 16 additions & 16 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
135 | 135 | | |
136 | 136 | | |
137 | 137 | | |
138 | | - | |
| 138 | + | |
139 | 139 | | |
140 | 140 | | |
141 | 141 | | |
142 | 142 | | |
143 | 143 | | |
144 | 144 | | |
145 | 145 | | |
146 | | - | |
| 146 | + | |
147 | 147 | | |
148 | 148 | | |
149 | 149 | | |
150 | 150 | | |
151 | 151 | | |
152 | 152 | | |
153 | 153 | | |
154 | | - | |
| 154 | + | |
155 | 155 | | |
156 | 156 | | |
157 | 157 | | |
| |||
199 | 199 | | |
200 | 200 | | |
201 | 201 | | |
202 | | - | |
| 202 | + | |
203 | 203 | | |
204 | 204 | | |
205 | 205 | | |
206 | 206 | | |
207 | 207 | | |
208 | 208 | | |
209 | 209 | | |
210 | | - | |
| 210 | + | |
211 | 211 | | |
212 | 212 | | |
213 | 213 | | |
214 | 214 | | |
215 | 215 | | |
216 | 216 | | |
217 | 217 | | |
218 | | - | |
| 218 | + | |
219 | 219 | | |
220 | 220 | | |
221 | 221 | | |
222 | 222 | | |
223 | 223 | | |
224 | 224 | | |
225 | 225 | | |
226 | | - | |
| 226 | + | |
227 | 227 | | |
228 | 228 | | |
229 | 229 | | |
230 | 230 | | |
231 | 231 | | |
232 | 232 | | |
233 | 233 | | |
234 | | - | |
| 234 | + | |
235 | 235 | | |
236 | 236 | | |
237 | 237 | | |
| |||
457 | 457 | | |
458 | 458 | | |
459 | 459 | | |
460 | | - | |
| 460 | + | |
461 | 461 | | |
462 | 462 | | |
463 | 463 | | |
464 | 464 | | |
465 | 465 | | |
466 | 466 | | |
467 | 467 | | |
468 | | - | |
| 468 | + | |
469 | 469 | | |
470 | 470 | | |
471 | 471 | | |
472 | 472 | | |
473 | 473 | | |
474 | 474 | | |
475 | 475 | | |
476 | | - | |
| 476 | + | |
477 | 477 | | |
478 | 478 | | |
479 | 479 | | |
| |||
521 | 521 | | |
522 | 522 | | |
523 | 523 | | |
524 | | - | |
| 524 | + | |
525 | 525 | | |
526 | 526 | | |
527 | 527 | | |
528 | 528 | | |
529 | 529 | | |
530 | 530 | | |
531 | 531 | | |
532 | | - | |
| 532 | + | |
533 | 533 | | |
534 | 534 | | |
535 | 535 | | |
536 | 536 | | |
537 | 537 | | |
538 | 538 | | |
539 | 539 | | |
540 | | - | |
| 540 | + | |
541 | 541 | | |
542 | 542 | | |
543 | 543 | | |
544 | 544 | | |
545 | 545 | | |
546 | 546 | | |
547 | 547 | | |
548 | | - | |
| 548 | + | |
549 | 549 | | |
550 | 550 | | |
551 | 551 | | |
552 | 552 | | |
553 | 553 | | |
554 | 554 | | |
555 | 555 | | |
556 | | - | |
| 556 | + | |
557 | 557 | | |
558 | 558 | | |
559 | 559 | | |
| |||
0 commit comments