[MINOR][PYSPARK][DOCS] Fix typo in example documentation #26299

mstill3 · 2019-10-29T18:22:29Z

What changes were proposed in this pull request?

I propose that we change the example code documentation to call the proper function .
For example, under the foreachBatch function, the example code was calling the foreach() function by mistake.

Why are the changes needed?

I suppose it could confuse some people, and it is a typo

Does this PR introduce any user-facing change?

No, there is no "meaningful" code being change, simply the documentation

How was this patch tested?

I made the change on a fork and it still worked

Under the `foreachBatch` function, the example code was calling the `foreach()` function by mistake

HyukjinKwon · 2019-10-30T00:44:30Z

ok to test

HyukjinKwon · 2019-10-30T00:44:47Z

Can you check if there are some more typos while we're here? I am sure there are more.

SparkQA · 2019-10-30T01:03:36Z

Test build #112873 has finished for PR 26299 at commit bd10f25.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Fixed minor documentation issues with grammar

Fixed slight documentation typo with cube function

Changed temp to temporary in documentation to conform with the rest of the documentation

vegerot · 2019-10-30T16:02:56Z

@HyukjinKwon He's just fixing some errors he's come across while going through the documentation. It would probably be best to take these fixes as they are now and to file more PRs later if he comes across them.

SparkQA · 2019-10-30T16:11:56Z

Test build #112931 has finished for PR 26299 at commit f7d77e7.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Change all default True and False values to consistent ``True`` or ``False``

SparkQA · 2019-10-30T16:17:21Z

Test build #112934 has finished for PR 26299 at commit 64cabce.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

The save method in this class did not have a proper example using the save function

SparkQA · 2019-10-30T16:33:38Z

Test build #112936 has finished for PR 26299 at commit bc61744.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

srowen

Yep several good catches in there

python/pyspark/sql/session.py

python/pyspark/sql/dataframe.py

python/pyspark/sql/context.py

Make the use of the word DataFrame consistent across documentation

SparkQA · 2019-10-30T16:58:27Z

Test build #112938 has finished for PR 26299 at commit c283d4f.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2019-10-30T17:06:23Z

Test build #112940 has finished for PR 26299 at commit 85dfe22.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

Some of the lines were > 100

SparkQA · 2019-10-30T18:21:13Z

Test build #112953 has finished for PR 26299 at commit b37f765.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2019-10-30T18:47:58Z

Test build #112957 has finished for PR 26299 at commit 9bbd649.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2019-10-30T19:38:59Z

Test build #112958 has finished for PR 26299 at commit 81d0855.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

python/pyspark/sql/readwriter.py

SparkQA · 2019-10-30T22:18:16Z

Test build #112969 has finished for PR 26299 at commit 43b6f2b.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2019-10-30T22:26:20Z

Test build #112970 has finished for PR 26299 at commit 4eb3537.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

<= 100 characters per line

SparkQA · 2019-10-30T23:03:09Z

Test build #112971 has finished for PR 26299 at commit cef3f12.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

python/pyspark/sql/readwriter.py

dongjoon-hyun

The UT failure seems to be the last blocker for this PR. Could you fix that in this PR?

update save() example code to match the DF columns

SparkQA · 2019-10-30T23:51:07Z

Test build #112973 has finished for PR 26299 at commit 0207d87.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

python/pyspark/sql/readwriter.py

Changed save mode to overwrite in save() documentation example

SparkQA · 2019-10-31T20:24:19Z

Test build #113045 has finished for PR 26299 at commit 0d5fc4d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

python/pyspark/sql/readwriter.py

HyukjinKwon

LGTM otherwise.

dongjoon-hyun · 2019-11-01T05:08:33Z

@mstill3 . Please update the PR to use os.path.join(tempfile.mkdtemp(), 'data' according to @HyukjinKwon 's advice.

dongjoon-hyun

LGTM for the others.

Update save() documentation example to create the file in the temp directory instead of in the root

SparkQA · 2019-11-01T16:01:35Z

Test build #113099 has finished for PR 26299 at commit 4e909e2.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

dongjoon-hyun

Thank you again for your first contribution, @mstill3 .
Thank you, @srowen, @HyukjinKwon , and @vegerot
Merged to master.

dongjoon-hyun · 2019-11-01T18:57:37Z

@mstill3 . Since this is useful for users, could you make a PR for branch-2.4 too?

Fix typo in example documentation

bd10f25

Under the `foreachBatch` function, the example code was calling the `foreach()` function by mistake

dongjoon-hyun added the DOCUMENTATION label Oct 30, 2019

mstill3 added 3 commits October 30, 2019 11:45

Fixed up some grammar issues

a17f9c9

Fixed minor documentation issues with grammar

Fixed typo

8830160

Fixed slight documentation typo with cube function

Fix typo

f7d77e7

Changed temp to temporary in documentation to conform with the rest of the documentation

Update default values

64cabce

Change all default True and False values to consistent ``True`` or ``False``

Fix documentation example typo

bc61744

The save method in this class did not have a proper example using the save function

srowen requested changes Oct 30, 2019

View reviewed changes

python/pyspark/sql/session.py Outdated Show resolved Hide resolved

python/pyspark/sql/session.py Outdated Show resolved Hide resolved

python/pyspark/sql/dataframe.py Show resolved Hide resolved

python/pyspark/sql/context.py Show resolved Hide resolved

Change all the data frames to DataFrame

c283d4f

Make the use of the word DataFrame consistent across documentation

mstill3 added 3 commits October 30, 2019 13:01

Update session.py

35afb15

Update session.py

9b29b76

Add spacing in front of new sentence line

85dfe22

Fixed my line length issues

b37f765

Some of the lines were > 100

Remove trailing whitespaces

9bbd649

mstill3 added 2 commits October 30, 2019 15:13

Fix note spacing issue

5525261

Remove improper indent

81d0855

dongjoon-hyun changed the title ~~Fix typo in example documentation~~ [MINOR][DOCS] Fix typo in example documentation Oct 30, 2019

dongjoon-hyun reviewed Oct 30, 2019

View reviewed changes

python/pyspark/sql/readwriter.py Outdated Show resolved Hide resolved

mstill3 added 2 commits October 30, 2019 18:20

Contistent use of DataFrames in documentation

529b4d5

Change the syntax back to how it was

4eb3537

Set documentation length to correct width

cef3f12

<= 100 characters per line

dongjoon-hyun reviewed Oct 30, 2019

View reviewed changes

python/pyspark/sql/readwriter.py Outdated Show resolved Hide resolved

dongjoon-hyun requested changes Oct 30, 2019

View reviewed changes

Fix example in documentation

0207d87

update save() example code to match the DF columns

mstill3 requested a review from dongjoon-hyun October 30, 2019 23:56

dongjoon-hyun reviewed Oct 31, 2019

View reviewed changes

python/pyspark/sql/readwriter.py Outdated Show resolved Hide resolved

Fix documentation example

0d5fc4d

Changed save mode to overwrite in save() documentation example

mstill3 requested a review from dongjoon-hyun October 31, 2019 22:13

HyukjinKwon reviewed Nov 1, 2019

View reviewed changes

python/pyspark/sql/readwriter.py Outdated Show resolved Hide resolved

HyukjinKwon approved these changes Nov 1, 2019

View reviewed changes

dongjoon-hyun approved these changes Nov 1, 2019

View reviewed changes

Change to saving in temp directory

4e909e2

Update save() documentation example to create the file in the temp directory instead of in the root

srowen approved these changes Nov 1, 2019

View reviewed changes

dongjoon-hyun approved these changes Nov 1, 2019

View reviewed changes

dongjoon-hyun closed this in 1e1b730 Nov 1, 2019

mstill3 deleted the patch-1 branch November 1, 2019 19:04

mstill3 restored the patch-1 branch November 1, 2019 19:04

mstill3 deleted the patch-1 branch November 1, 2019 19:04

mstill3 restored the patch-1 branch November 1, 2019 19:05

[MINOR][PYSPARK][DOCS] Fix typo in example documentation #26299

[MINOR][PYSPARK][DOCS] Fix typo in example documentation #26299

Uh oh!

Conversation

mstill3 commented Oct 29, 2019 • edited by dongjoon-hyun Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Uh oh!

HyukjinKwon commented Oct 30, 2019

Uh oh!

HyukjinKwon commented Oct 30, 2019

Uh oh!

SparkQA commented Oct 30, 2019

Uh oh!

vegerot commented Oct 30, 2019

Uh oh!

SparkQA commented Oct 30, 2019

Uh oh!

SparkQA commented Oct 30, 2019

Uh oh!

SparkQA commented Oct 30, 2019

Uh oh!

srowen left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

SparkQA commented Oct 30, 2019

Uh oh!

SparkQA commented Oct 30, 2019

Uh oh!

SparkQA commented Oct 30, 2019

Uh oh!

SparkQA commented Oct 30, 2019

Uh oh!

SparkQA commented Oct 30, 2019

Uh oh!

Uh oh!

SparkQA commented Oct 30, 2019

Uh oh!

SparkQA commented Oct 30, 2019

Uh oh!

SparkQA commented Oct 30, 2019

Uh oh!

Uh oh!

dongjoon-hyun left a comment

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Oct 30, 2019

Uh oh!

Uh oh!

SparkQA commented Oct 31, 2019

Uh oh!

Uh oh!

HyukjinKwon left a comment

Choose a reason for hiding this comment

Uh oh!

dongjoon-hyun commented Nov 1, 2019

Uh oh!

dongjoon-hyun left a comment

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Nov 1, 2019

Uh oh!

dongjoon-hyun left a comment

Choose a reason for hiding this comment

Uh oh!

dongjoon-hyun commented Nov 1, 2019

Uh oh!

Uh oh!

mstill3 commented Oct 29, 2019 •

edited by dongjoon-hyun

Loading