Skip to content

Commit

Permalink
[SPARK-50711][PS][DOCS] Upgrade the minimum version of Pandas in PS t…
Browse files Browse the repository at this point in the history
…o 2.2.0

### What changes were proposed in this pull request?
Upgrade the minimum version of Pandas in PS to 2.2.0

But the version used in `require_minimum_pandas_version` is still 2.0.0, because currently this function is used in both PS and non-PS modules.

### Why are the changes needed?
Actually, PS with Pandas < 2.2.0 has already been broken in master branch

### Does this PR introduce _any_ user-facing change?
No, doc-only

### How was this patch tested?
CI

### Was this patch authored or co-authored using generative AI tooling?
No

Closes #49344 from zhengruifeng/ps_pd_mini.

Authored-by: Ruifeng Zheng <ruifengz@apache.org>
Signed-off-by: Ruifeng Zheng <ruifengz@apache.org>
  • Loading branch information
zhengruifeng committed Jan 2, 2025
1 parent 5c63484 commit 3c0824d
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 2 deletions.
2 changes: 1 addition & 1 deletion dev/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ py4j>=0.10.9.7
numpy>=1.21
pyarrow>=11.0.0
six==1.16.0
pandas>=2.0.0
pandas>=2.2.0
scipy
plotly>=4.8
mlflow>=2.3.1
Expand Down
2 changes: 1 addition & 1 deletion python/docs/source/getting_started/install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -239,7 +239,7 @@ Installable with ``pip install "pyspark[pandas_on_spark]"``.
========= ================= ================================
Package Supported version Note
========= ================= ================================
`pandas` >=2.0.0 Required for Pandas API on Spark
`pandas` >=2.2.0 Required for Pandas API on Spark
`pyarrow` >=11.0.0 Required for Pandas API on Spark
========= ================= ================================

Expand Down

0 comments on commit 3c0824d

Please sign in to comment.