Commit 16ec7a0
[SPARK-42338][CONNECT] Add details to non-fatal errors to raise a proper exception in the Python client
### What changes were proposed in this pull request?
Adds details to non-fatal errors to raise a proper exception in the Python client, which makes `df.sample` raise `IllegalArgumentException` as same as PySpark, except for the timing that is delayed to call actions.
### Why are the changes needed?
Currently `SparkConnectService` does not add details for `NonFatal` exceptions to the `PRCStatus`, so the Python client can't detect the exception properly and raises `SparkConnectGrpcException` instead.
It also should have the details for the Python client.
### Does this PR introduce _any_ user-facing change?
Users will see a proper exception when they call `df.sample` with illegal arguments, but in a different timing.
### How was this patch tested?
Enabled `DataFrameParityTests.test_sample`.
Closes apache#39957 from ueshin/issues/SPARK-42338/sample.
Authored-by: Takuya UESHIN <ueshin@databricks.com>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
(cherry picked from commit ced6750)
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>1 parent e7d4a75 commit 16ec7a0
File tree
2 files changed
+8
-2
lines changed- connector/connect/server/src/main/scala/org/apache/spark/sql/connect/service
- python/pyspark/sql/tests/connect
2 files changed
+8
-2
lines changedLines changed: 7 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
109 | 109 | | |
110 | 110 | | |
111 | 111 | | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
112 | 119 | | |
113 | 120 | | |
114 | 121 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
85 | 85 | | |
86 | 86 | | |
87 | 87 | | |
88 | | - | |
89 | | - | |
| 88 | + | |
90 | 89 | | |
91 | 90 | | |
92 | 91 | | |
| |||
0 commit comments