Skip to content

Commit 9eccefd

Browse files
committed
ARROW-357: Use a single RowGroup for Parquet files as default.
Change-Id: Ibdbd1db9fcd6c2e6ce588b3f326caf00d38df48a
1 parent 772bc6e commit 9eccefd

File tree

1 file changed

+3
-2
lines changed

1 file changed

+3
-2
lines changed

python/pyarrow/parquet.pyx

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -106,7 +106,8 @@ def write_table(table, filename, chunk_size=None, version=None,
106106
table : pyarrow.Table
107107
filename : string
108108
chunk_size : int
109-
The maximum number of rows in each Parquet RowGroup
109+
The maximum number of rows in each Parquet RowGroup. As a default,
110+
we will write a single RowGroup per file.
110111
version : {"1.0", "2.0"}, default "1.0"
111112
The Parquet format version, defaults to 1.0
112113
use_dictionary : bool or list
@@ -121,7 +122,7 @@ def write_table(table, filename, chunk_size=None, version=None,
121122
cdef WriterProperties.Builder properties_builder
122123
cdef int64_t chunk_size_ = 0
123124
if chunk_size is None:
124-
chunk_size_ = min(ctable_.num_rows(), int(2**16))
125+
chunk_size_ = ctable_.num_rows()
125126
else:
126127
chunk_size_ = chunk_size
127128

0 commit comments

Comments
 (0)