Skip to content

[SPARK-35509][DOCS] Move text data source options from Python and Scala into a single page #32660

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 4 commits into from

Conversation

itholic
Copy link
Contributor

@itholic itholic commented May 25, 2021

What changes were proposed in this pull request?

This PR proposes move text data source options from Python, Scala and Java into a single page.

Why are the changes needed?

So far, the documentation for text data source options is separated into different pages for each language API documents. However, this makes managing many options inconvenient, so it is efficient to manage all options in a single page and provide a link to that page in the API of each language.

Does this PR introduce any user-facing change?

Yes, the documents will be shown below after this change:

  • "Text Files" page

Screen Shot 2021-05-26 at 3 20 11 PM

  • Python

Screen Shot 2021-05-25 at 5 04 26 PM

  • Scala

Screen Shot 2021-05-25 at 5 05 10 PM

  • Java

Screen Shot 2021-05-25 at 5 05 36 PM

How was this patch tested?

Manually build docs and confirm the page.

@SparkQA
Copy link

SparkQA commented May 25, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43436/

@SparkQA
Copy link

SparkQA commented May 25, 2021

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43436/

@SparkQA
Copy link

SparkQA commented May 25, 2021

Test build #138915 has finished for PR 32660 at commit cddc556.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented May 26, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43469/

@SparkQA
Copy link

SparkQA commented May 26, 2021

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43469/

@SparkQA
Copy link

SparkQA commented May 26, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43472/

@SparkQA
Copy link

SparkQA commented May 26, 2021

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43472/

Copy link
Member

@HyukjinKwon HyukjinKwon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM otherwise

@SparkQA
Copy link

SparkQA commented May 26, 2021

Test build #138949 has finished for PR 32660 at commit 1ee72e2.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented May 26, 2021

Test build #138952 has finished for PR 32660 at commit 98840f3.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented May 26, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43484/

@SparkQA
Copy link

SparkQA commented May 26, 2021

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43484/

@HyukjinKwon
Copy link
Member

Merged to master.

* `DataFrameWriter`
* `DataStreamReader`
* `DataStreamWriter`
* `OPTIONS` clause at [CREATE TABLE USING DATA_SOURCE](sql-ref-syntax-ddl-create-table-datasource.html)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I missed it here. This is not in "the .option/.options methods of". Can you make the level down?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. let me fix it in #32658

@SparkQA
Copy link

SparkQA commented May 26, 2021

Test build #138965 has finished for PR 32660 at commit f03c03d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants