Skip to content

Commit 4e0175f

Browse files
committed
Fixes Python Parquet API, we need Py4J array to call varargs method
1 parent 0d8ec1d commit 4e0175f

File tree

1 file changed

+5
-1
lines changed

1 file changed

+5
-1
lines changed

python/pyspark/sql.py

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1483,7 +1483,11 @@ def parquetFile(self, *paths):
14831483
>>> sorted(df.collect()) == sorted(df2.collect())
14841484
True
14851485
"""
1486-
jdf = self._ssql_ctx.parquetFile(*path)
1486+
gateway = self._sc._gateway
1487+
jpaths = gateway.new_array(gateway.jvm.java.lang.String, len(paths))
1488+
for i in range(0, len(paths)):
1489+
jpaths[i] = paths[i]
1490+
jdf = self._ssql_ctx.parquetFile(jpaths)
14871491
return DataFrame(jdf, self)
14881492

14891493
def jsonFile(self, path, schema=None, samplingRatio=1.0):

0 commit comments

Comments
 (0)