Skip to content

Commit

Permalink
[SPARK-35250][SQL][DOCS] Fix duplicated STOP_AT_DELIMITER to SKIP_VAL…
Browse files Browse the repository at this point in the history
…UE at CSV's unescapedQuoteHandling option documentation

### What changes were proposed in this pull request?

This is rather a followup of apache#30518 that should be ported back to `branch-3.1` too.
`STOP_AT_DELIMITER` was mistakenly used twice. The duplicated `STOP_AT_DELIMITER` should be `SKIP_VALUE` in the documentation.

### Why are the changes needed?

To correctly document.

### Does this PR introduce _any_ user-facing change?

Yes, it fixes the user-facing documentation.

### How was this patch tested?

I checked them via running linters.

Closes apache#32423 from HyukjinKwon/SPARK-35250.

Authored-by: HyukjinKwon <gurwls223@apache.org>
Signed-off-by: HyukjinKwon <gurwls223@apache.org>
  • Loading branch information
HyukjinKwon committed May 3, 2021
1 parent 54e0aa1 commit 8aaa9e8
Show file tree
Hide file tree
Showing 4 changed files with 4 additions and 4 deletions.
2 changes: 1 addition & 1 deletion python/pyspark/sql/readwriter.py
Original file line number Diff line number Diff line change
Expand Up @@ -728,7 +728,7 @@ def csv(self, path, schema=None, sep=None, encoding=None, quote=None, escape=Non
* ``STOP_AT_DELIMITER``: If unescaped quotes are found in the input, consider the value
as an unquoted value. This will make the parser accumulate all characters until the
delimiter or a line ending is found in the input.
* ``STOP_AT_DELIMITER``: If unescaped quotes are found in the input, the content parsed
* ``SKIP_VALUE``: If unescaped quotes are found in the input, the content parsed
for the given value will be skipped and the value set in nullValue will be produced
instead.
* ``RAISE_ERROR``: If unescaped quotes are found in the input, a TextParsingException
Expand Down
2 changes: 1 addition & 1 deletion python/pyspark/sql/streaming.py
Original file line number Diff line number Diff line change
Expand Up @@ -941,7 +941,7 @@ def csv(self, path, schema=None, sep=None, encoding=None, quote=None, escape=Non
* ``STOP_AT_DELIMITER``: If unescaped quotes are found in the input, consider the value
as an unquoted value. This will make the parser accumulate all characters until the
delimiter or a line ending is found in the input.
* ``STOP_AT_DELIMITER``: If unescaped quotes are found in the input, the content parsed
* ``SKIP_VALUE``: If unescaped quotes are found in the input, the content parsed
for the given value will be skipped and the value set in nullValue will be produced
instead.
* ``RAISE_ERROR``: If unescaped quotes are found in the input, a TextParsingException
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -747,7 +747,7 @@ class DataFrameReader private[sql](sparkSession: SparkSession) extends Logging {
* <li>`STOP_AT_DELIMITER`: If unescaped quotes are found in the input, consider the value
* as an unquoted value. This will make the parser accumulate all characters until the
* delimiter or a line ending is found in the input.</li>
* <li>`STOP_AT_DELIMITER`: If unescaped quotes are found in the input, the content parsed
* <li>`SKIP_VALUE`: If unescaped quotes are found in the input, the content parsed
* for the given value will be skipped and the value set in nullValue will be produced
* instead.</li>
* <li>`RAISE_ERROR`: If unescaped quotes are found in the input, a TextParsingException
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -412,7 +412,7 @@ final class DataStreamReader private[sql](sparkSession: SparkSession) extends Lo
* <li>`STOP_AT_DELIMITER`: If unescaped quotes are found in the input, consider the value
* as an unquoted value. This will make the parser accumulate all characters until the
* delimiter or a line ending is found in the input.</li>
* <li>`STOP_AT_DELIMITER`: If unescaped quotes are found in the input, the content parsed
* <li>`SKIP_VALUE`: If unescaped quotes are found in the input, the content parsed
* for the given value will be skipped and the value set in nullValue will be produced
* instead.</li>
* <li>`RAISE_ERROR`: If unescaped quotes are found in the input, a TextParsingException
Expand Down

0 comments on commit 8aaa9e8

Please sign in to comment.