Skip to content

Commit 44922c5

Browse files
srowenJackey Lee
authored andcommitted
[SPARK-25760][DOCS][FOLLOWUP] Add note about AddJar return value change in migration guide
## What changes were proposed in this pull request? Add note about AddJar return value change in migration guide ## How was this patch tested? n/a Closes apache#22826 from srowen/SPARK-25760.2. Authored-by: Sean Owen <sean.owen@databricks.com> Signed-off-by: Sean Owen <sean.owen@databricks.com>
1 parent efa824a commit 44922c5

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

docs/sql-migration-guide-upgrade.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,8 @@ displayTitle: Spark SQL Upgrading Guide
1515

1616
- Since Spark 3.0, the `from_json` functions supports two modes - `PERMISSIVE` and `FAILFAST`. The modes can be set via the `mode` option. The default mode became `PERMISSIVE`. In previous versions, behavior of `from_json` did not conform to either `PERMISSIVE` nor `FAILFAST`, especially in processing of malformed JSON records. For example, the JSON string `{"a" 1}` with the schema `a INT` is converted to `null` by previous versions but Spark 3.0 converts it to `Row(null)`.
1717

18+
- The `ADD JAR` command previously returned a result set with the single value 0. It now returns an empty result set.
19+
1820
## Upgrading From Spark SQL 2.3 to 2.4
1921

2022
- In Spark version 2.3 and earlier, the second parameter to array_contains function is implicitly promoted to the element type of first array type parameter. This type promotion can be lossy and may cause `array_contains` function to return wrong result. This problem has been addressed in 2.4 by employing a safer type promotion mechanism. This can cause some change in behavior and are illustrated in the table below.

0 commit comments

Comments
 (0)