Skip to content

Commit 44eba46

Browse files
codesorceryHyukjinKwon
authored andcommitted
[SPARK-48710][PYTHON][3.5] Limit NumPy version to supported range (>=1.15,<2)
### What changes were proposed in this pull request? * Add a constraint for `numpy<2` to the PySpark package ### Why are the changes needed? PySpark references some code which was removed with NumPy 2.0. Thus, if `numpy>=2` is installed, executing PySpark may fail. #47083 updates the `master` branch to be compatible with NumPy 2. This PR adds a version bound for older releases, where it won't be applied. ### Does this PR introduce _any_ user-facing change? NumPy will be limited to `numpy<2` when installing `pypspark` with extras `ml`, `mllib`, `sql`, `pandas_on_spark` or `connect`. ### How was this patch tested? Via existing CI jobs. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #47175 from codesorcery/SPARK-48710-numpy-upper-bound. Authored-by: Patrick Marx <6949483+codesorcery@users.noreply.github.com> Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
1 parent ef4e456 commit 44eba46

File tree

1 file changed

+5
-5
lines changed

1 file changed

+5
-5
lines changed

python/setup.py

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -307,25 +307,25 @@ def run(self):
307307
# if you're updating the versions or dependencies.
308308
install_requires=["py4j==0.10.9.7"],
309309
extras_require={
310-
"ml": ["numpy>=1.15"],
311-
"mllib": ["numpy>=1.15"],
310+
"ml": ["numpy>=1.15,<2"],
311+
"mllib": ["numpy>=1.15,<2"],
312312
"sql": [
313313
"pandas>=%s" % _minimum_pandas_version,
314314
"pyarrow>=%s" % _minimum_pyarrow_version,
315-
"numpy>=1.15",
315+
"numpy>=1.15,<2",
316316
],
317317
"pandas_on_spark": [
318318
"pandas>=%s" % _minimum_pandas_version,
319319
"pyarrow>=%s" % _minimum_pyarrow_version,
320-
"numpy>=1.15",
320+
"numpy>=1.15,<2",
321321
],
322322
"connect": [
323323
"pandas>=%s" % _minimum_pandas_version,
324324
"pyarrow>=%s" % _minimum_pyarrow_version,
325325
"grpcio>=%s" % _minimum_grpc_version,
326326
"grpcio-status>=%s" % _minimum_grpc_version,
327327
"googleapis-common-protos>=%s" % _minimum_googleapis_common_protos_version,
328-
"numpy>=1.15",
328+
"numpy>=1.15,<2",
329329
],
330330
},
331331
python_requires=">=3.8",

0 commit comments

Comments
 (0)