Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Start migrating I/O writers to pylibcudf (starting with JSON) #15952

Merged
merged 38 commits into from
Jul 2, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
591cdd2
Start migrating I/O writers to pylibcudf (starting with JSON)
lithomas1 Jun 6, 2024
15daaaa
update docs
lithomas1 Jun 7, 2024
72204f1
Merge branch 'branch-24.08' of github.com:rapidsai/cudf into pylibcud…
lithomas1 Jun 7, 2024
c24664c
update and start writing tests
lithomas1 Jun 7, 2024
8c88c7c
Merge branch 'branch-24.08' of github.com:rapidsai/cudf into pylibcud…
lithomas1 Jun 11, 2024
2b3853f
add some tests
lithomas1 Jun 11, 2024
cd6df5e
Merge branch 'branch-24.08' of github.com:rapidsai/cudf into pylibcud…
lithomas1 Jun 11, 2024
c54316e
update
lithomas1 Jun 11, 2024
dc93356
Merge branch 'branch-24.08' of github.com:rapidsai/cudf into pylibcud…
lithomas1 Jun 12, 2024
8c4c4e4
address comments
lithomas1 Jun 12, 2024
63358e9
Merge branch 'branch-24.08' of github.com:rapidsai/cudf into pylibcud…
lithomas1 Jun 12, 2024
9150a6c
try something else
lithomas1 Jun 12, 2024
b1951d0
try fix
lithomas1 Jun 13, 2024
1228569
update following feedback
lithomas1 Jun 13, 2024
699efd3
cleanup tests
lithomas1 Jun 13, 2024
e242182
address more comments
lithomas1 Jun 13, 2024
564358f
Merge branch 'branch-24.08' into pylibcudf-io-writers
lithomas1 Jun 17, 2024
e0901dd
fix bad merge
lithomas1 Jun 17, 2024
d22953f
Merge branch 'branch-24.08' into pylibcudf-io-writers
lithomas1 Jun 18, 2024
604c16d
address more comments
lithomas1 Jun 24, 2024
e6c3ec7
address more comments
lithomas1 Jun 24, 2024
624d444
fix all nested struct cases
lithomas1 Jun 24, 2024
53b821c
Merge branch 'pylibcudf-io-writers' of github.com:lithomas1/cudf into…
lithomas1 Jun 24, 2024
186a2fb
Merge branch 'branch-24.08' of github.com:rapidsai/cudf into pylibcud…
lithomas1 Jun 24, 2024
9a6a896
Merge branch 'branch-24.08' of github.com:rapidsai/cudf into pylibcud…
lithomas1 Jun 25, 2024
0ed9af6
Fix error in testing utils
lithomas1 Jun 25, 2024
aff6178
small test fixes
lithomas1 Jun 25, 2024
f7cd9e6
cleanup utils
lithomas1 Jun 26, 2024
c5a3fbe
Merge branch 'branch-24.08' of github.com:rapidsai/cudf into pylibcud…
lithomas1 Jun 27, 2024
79c1dfd
clean source_or_sink
lithomas1 Jun 27, 2024
8fc139f
Merge branch 'pylibcudf-io-writers' of github.com:lithomas1/cudf into…
lithomas1 Jun 27, 2024
e940e30
Address code review
lithomas1 Jun 27, 2024
e57a677
Merge branch 'branch-24.08' of github.com:rapidsai/cudf into pylibcud…
lithomas1 Jun 29, 2024
7806ce4
simplify again
lithomas1 Jun 29, 2024
25c25d4
Merge branch 'branch-24.08' of github.com:rapidsai/cudf into pylibcud…
lithomas1 Jul 1, 2024
60287e1
address more comments
lithomas1 Jul 1, 2024
205c32c
Update python/cudf/cudf/_lib/pylibcudf/io/types.pyx
vyasr Jul 2, 2024
d325b64
Merge branch 'branch-24.08' into pylibcudf-io-writers
vyasr Jul 2, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
update docs
  • Loading branch information
lithomas1 committed Jun 7, 2024
commit 15daaaac9290307bc2153e379627548994e277fe
6 changes: 6 additions & 0 deletions docs/cudf/source/user_guide/api_docs/pylibcudf/io/json.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
====
JSON
====

.. automodule:: cudf._lib.pylibcudf.io.json
:members:
4 changes: 2 additions & 2 deletions python/cudf/cudf/_lib/pylibcudf/io/avro.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ cpdef TableWithMetadata read_avro(
size_type num_rows = -1
):
"""
Reads an Avro dataset into a set of columns.
Reads an Avro dataset into a :py:class:`~.types.TableWithMetadata`.

Parameters
----------
Expand All @@ -36,7 +36,7 @@ cpdef TableWithMetadata read_avro(
Returns
-------
TableWithMetadata
The Table and its corresponding metadata that was read in.
The Table and its corresponding metadata (column names) that were read in.
"""
cdef vector[string] c_columns
if columns is not None and len(columns) > 0:
Expand Down
20 changes: 20 additions & 0 deletions python/cudf/cudf/_lib/pylibcudf/io/json.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,26 @@ cpdef void write_json(
str false_value = "false"
):
"""
Writes a :py:class:`~cudf._lib.pylibcudf.types.Table` to JSON format.

Parameters
----------
sink_info: SinkInfo
The SinkInfo object to write the JSON to.
table_w_meta: TableWithMetadata
The TableWithMetadata object containing the Table to write
na_rep: str, default ""
The string representation for null values.
include_nulls: bool, default False
Enables/Disables output of nulls as 'null'.
lines: bool, default False
If `True`, write output in the JSON lines format.
rows_per_chunk: int, default 2,147,483,647
The maximum number of rows to write at a time.
true_value: str, default "true"
The string representation for values != 0 in INT8 types.
false_value: str, default "false"
The string representation for values == 0 in INT8 types.
"""
cdef table_metadata tbl_meta = table_w_meta.metadata
cdef string na_rep_c = na_rep.encode()
Expand Down
Empty file.