Skip to content

Commit 89f5ec7

Browse files
committed
[SPARK-35250][SQL][DOCS] Fix duplicated STOP_AT_DELIMITER to SKIP_VALUE at CSV's unescapedQuoteHandling option documentation
### What changes were proposed in this pull request? This is rather a followup of #30518 that should be ported back to `branch-3.1` too. `STOP_AT_DELIMITER` was mistakenly used twice. The duplicated `STOP_AT_DELIMITER` should be `SKIP_VALUE` in the documentation. ### Why are the changes needed? To correctly document. ### Does this PR introduce _any_ user-facing change? Yes, it fixes the user-facing documentation. ### How was this patch tested? I checked them via running linters. Closes #32423 from HyukjinKwon/SPARK-35250. Authored-by: HyukjinKwon <gurwls223@apache.org> Signed-off-by: HyukjinKwon <gurwls223@apache.org> (cherry picked from commit 8aaa9e8) Signed-off-by: HyukjinKwon <gurwls223@apache.org>
1 parent 25002e0 commit 89f5ec7

File tree

4 files changed

+4
-4
lines changed

4 files changed

+4
-4
lines changed

python/pyspark/sql/readwriter.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -701,7 +701,7 @@ def csv(self, path, schema=None, sep=None, encoding=None, quote=None, escape=Non
701701
* ``STOP_AT_DELIMITER``: If unescaped quotes are found in the input, consider the value
702702
as an unquoted value. This will make the parser accumulate all characters until the
703703
delimiter or a line ending is found in the input.
704-
* ``STOP_AT_DELIMITER``: If unescaped quotes are found in the input, the content parsed
704+
* ``SKIP_VALUE``: If unescaped quotes are found in the input, the content parsed
705705
for the given value will be skipped and the value set in nullValue will be produced
706706
instead.
707707
* ``RAISE_ERROR``: If unescaped quotes are found in the input, a TextParsingException

python/pyspark/sql/streaming.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -915,7 +915,7 @@ def csv(self, path, schema=None, sep=None, encoding=None, quote=None, escape=Non
915915
* ``STOP_AT_DELIMITER``: If unescaped quotes are found in the input, consider the value
916916
as an unquoted value. This will make the parser accumulate all characters until the
917917
delimiter or a line ending is found in the input.
918-
* ``STOP_AT_DELIMITER``: If unescaped quotes are found in the input, the content parsed
918+
* ``SKIP_VALUE``: If unescaped quotes are found in the input, the content parsed
919919
for the given value will be skipped and the value set in nullValue will be produced
920920
instead.
921921
* ``RAISE_ERROR``: If unescaped quotes are found in the input, a TextParsingException

sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -743,7 +743,7 @@ class DataFrameReader private[sql](sparkSession: SparkSession) extends Logging {
743743
* <li>`STOP_AT_DELIMITER`: If unescaped quotes are found in the input, consider the value
744744
* as an unquoted value. This will make the parser accumulate all characters until the
745745
* delimiter or a line ending is found in the input.</li>
746-
* <li>`STOP_AT_DELIMITER`: If unescaped quotes are found in the input, the content parsed
746+
* <li>`SKIP_VALUE`: If unescaped quotes are found in the input, the content parsed
747747
* for the given value will be skipped and the value set in nullValue will be produced
748748
* instead.</li>
749749
* <li>`RAISE_ERROR`: If unescaped quotes are found in the input, a TextParsingException

sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamReader.scala

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -412,7 +412,7 @@ final class DataStreamReader private[sql](sparkSession: SparkSession) extends Lo
412412
* <li>`STOP_AT_DELIMITER`: If unescaped quotes are found in the input, consider the value
413413
* as an unquoted value. This will make the parser accumulate all characters until the
414414
* delimiter or a line ending is found in the input.</li>
415-
* <li>`STOP_AT_DELIMITER`: If unescaped quotes are found in the input, the content parsed
415+
* <li>`SKIP_VALUE`: If unescaped quotes are found in the input, the content parsed
416416
* for the given value will be skipped and the value set in nullValue will be produced
417417
* instead.</li>
418418
* <li>`RAISE_ERROR`: If unescaped quotes are found in the input, a TextParsingException

0 commit comments

Comments
 (0)