Skip to content

Commit 3d334ac

Browse files
rahulsmahadevsrowen
authored andcommitted
[SPARK-29494][SQL][2.4] Fix for ArrayOutofBoundsException while converting string to timestamp
### What changes were proposed in this pull request? * Adding an additional check in `stringToTimestamp` to handle cases where the input has trailing ':' * Added a test to make sure this works. ### Why are the changes needed? In a couple of scenarios while converting from String to Timestamp `DateTimeUtils.stringToTimestamp` throws an array out of bounds exception if there is trailing ':'. The behavior of this method requires it to return `None` in case the format of the string is incorrect. ### Does this PR introduce any user-facing change? No ### How was this patch tested? Added a test in the `DateTimeTestUtils` suite to test if my fix works. Closes #26171 from rahulsmahadev/araryOB. Authored-by: Rahul Mahadev <rahul.mahadev@databricks.com> Signed-off-by: Sean Owen <sean.owen@databricks.com>
1 parent b094774 commit 3d334ac

File tree

2 files changed

+7
-1
lines changed

2 files changed

+7
-1
lines changed

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -378,7 +378,7 @@ object DateTimeUtils {
378378
i += 1
379379
}
380380
} else {
381-
if (b == ':' || b == ' ') {
381+
if (i < segments.length && (b == ':' || b == ' ')) {
382382
segments(i) = currentSegmentValue
383383
currentSegmentValue = 0
384384
i += 1

sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeUtilsSuite.scala

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -581,6 +581,12 @@ class DateTimeUtilsSuite extends SparkFunSuite {
581581
}
582582
}
583583

584+
test("trailing characters while converting string to timestamp") {
585+
val s = UTF8String.fromString("2019-10-31T10:59:23Z:::")
586+
val time = DateTimeUtils.stringToTimestamp(s, DateTimeUtils.defaultTimeZone())
587+
assert(time == None)
588+
}
589+
584590
test("truncTimestamp") {
585591
def testTrunc(
586592
level: Int,

0 commit comments

Comments
 (0)