Skip to content

Commit 441e15e

Browse files
itholicdongjoon-hyun
authored andcommitted
[SPARK-46087][PYTHON] Sync PySpark dependencies in docs and dev requirements
### What changes were proposed in this pull request? This PR proposes to synchronize the versions of dependencies listed in the [PySpark documentation](https://spark.apache.org/docs/latest/api/python/getting_started/install.html#dependencies) with those specified in the [dev/requirements.txt](https://github.com/apache/spark/blob/master/dev/requirements.txt) file. ### Why are the changes needed? Aligning the versions of dependencies ensures that the development environment reflects the actual user environment more accurately. ### Does this PR introduce _any_ user-facing change? No API changes. ### How was this patch tested? Build the documents from latest master branch manually and sync the version of dependencies: <img width="774" alt="Screenshot 2023-11-24 at 2 49 09 PM" src="https://github.com/apache/spark/assets/44108233/4f539fc7-0bbc-4fe3-949f-b4c225122b56"> ### Was this patch authored or co-authored using generative AI tooling? No. Closes apache#44000 from itholic/req-sync. Authored-by: Haejoon Lee <haejoon.lee@databricks.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
1 parent 4530b12 commit 441e15e

File tree

1 file changed

+6
-6
lines changed

1 file changed

+6
-6
lines changed

dev/requirements.txt

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
11
# PySpark dependencies (required)
2-
py4j
2+
py4j>=0.10.9.7
33

44
# PySpark dependencies (optional)
5-
numpy
6-
pyarrow
5+
numpy>=1.21
6+
pyarrow>=4.0.0
77
six==1.16.0
8-
pandas
8+
pandas>=1.4.4
99
scipy
1010
plotly
1111
mlflow>=2.3.1
@@ -52,8 +52,8 @@ black==23.9.1
5252
py
5353

5454
# Spark Connect (required)
55-
grpcio==1.59.3
56-
grpcio-status==1.59.3
55+
grpcio>=1.59.3
56+
grpcio-status>=1.59.3
5757
protobuf==4.25.1
5858
googleapis-common-protos>=1.56.4
5959

0 commit comments

Comments
 (0)