Minor: Improve DataFrame docs, add examples
#9159
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
alamb
added
documentation
alamb
commented
Feb 8, 2024
| /// ).await?; | ||
| /// # Ok(()) | ||
| /// # } | ||
| /// ``` | ||
| pub async fn write_parquet( |
There was a problem hiding this comment.
This is the example I wanted to write, but I started clicking through the API to get here and I saw too much along the way that needed to be fixed too, so I got a bit carried away and worked on quite a bit of the docs
github-actions
bot
added
core
carols10cents
approved these changes
Feb 8, 2024
| /// Convert the logical plan represented by this DataFrame into a physical plan and | ||
| /// execute it, collecting all resulting batches into memory | ||
| /// Executes this DataFrame and collects all results into a vector of RecordBatch. | ||
| /// Execute this `DataFrame` and buffer all resulting `RecordBatch`es into memory. |
There was a problem hiding this comment.
This change in particular is excellent! The reason I'd want to call this method is precisely because I don't want to think about logical plans and physical plans and such 😉
| /// Note: This method should not be used outside testing, as it loses the | ||
| /// snapshot of the [`SessionState`] attached to this [`DataFrame`] and | ||
| /// consequently subsequent operations may take place against a different | ||
| /// state (e.g. a different value of `now()`) |
| @@ -1025,6 +1107,7 @@ impl DataFrame { | |||
| } | |||
|
|
|||
| /// Write this DataFrame to the referenced table by name. | |||
| /// | |||
| /// This method uses on the same underlying implementation | |||
There was a problem hiding this comment.
This line wasn't changed in this PR, but it caught my eye... it seems like there's an extra word?
Suggested change
| /// This method uses on the same underlying implementation | |
| /// This method uses the same underlying implementation |
There was a problem hiding this comment.
I missed this comment - will make a follow on PR
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
|
Thank you for the review @carols10cents 🙏 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Part of #7013
Rationale for this change
I was watching @carols10cents try to find how to write a parquet file via a
DataFramein the API docs and they left something to be desiredWhat changes are included in this PR?
DataFrame::write_json,write_csv,write_parquetAre these changes tested?
Covered by CI doc tests
Are there any user-facing changes?
Better docs