-
Notifications
You must be signed in to change notification settings - Fork 28.6k
Commit 224d3ba
[SPARK-49986][INFRA] Restore
### What changes were proposed in this pull request?
Restore `scipy` installation in dockerfile
### Why are the changes needed?
https://docs.scipy.org/doc/scipy-1.13.1/building/index.html#system-level-dependencies
> If you want to use the system Python and pip, you will need:
C, C++, and Fortran compilers (typically gcc, g++, and gfortran).
...
`scipy` actually depends on `gfortran`, but `apt-get remove --purge -y 'gfortran-11'` broke this dependency.
### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
manually check with the first commit 5be0dfa:
move `apt-get remove --purge -y 'gfortran-11'` ahead of `scipy` installation, then the installation fails with
```
#18 394.3 Collecting scipy
#18 394.4 Downloading scipy-1.13.1.tar.gz (57.2 MB)
#18 395.2 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.2/57.2 MB 76.7 MB/s eta 0:00:00
#18 401.3 Installing build dependencies: started
#18 410.5 Installing build dependencies: finished with status 'done'
#18 410.5 Getting requirements to build wheel: started
#18 410.7 Getting requirements to build wheel: finished with status 'done'
#18 410.7 Installing backend dependencies: started
#18 411.8 Installing backend dependencies: finished with status 'done'
#18 411.8 Preparing metadata (pyproject.toml): started
#18 414.9 Preparing metadata (pyproject.toml): finished with status 'error'
#18 414.9 error: subprocess-exited-with-error
#18 414.9
#18 414.9 × Preparing metadata (pyproject.toml) did not run successfully.
#18 414.9 │ exit code: 1
#18 414.9 ╰─> [42 lines of output]
#18 414.9 + meson setup /tmp/pip-install-y77ar9d0/scipy_1e543e0816ed4b26984415533ae9079d /tmp/pip-install-y77ar9d0/scipy_1e543e0816ed4b26984415533ae9079d/.mesonpy-xqfvs4ek -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --native-file=/tmp/pip-install-y77ar9d0/scipy_1e543e0816ed4b26984415533ae9079d/.mesonpy-xqfvs4ek/meson-python-native-file.ini
#18 414.9 The Meson build system
#18 414.9 Version: 1.5.2
#18 414.9 Source dir: /tmp/pip-install-y77ar9d0/scipy_1e543e0816ed4b26984415533ae9079d
#18 414.9 Build dir: /tmp/pip-install-y77ar9d0/scipy_1e543e0816ed4b26984415533ae9079d/.mesonpy-xqfvs4ek
#18 414.9 Build type: native build
#18 414.9 Project name: scipy
#18 414.9 Project version: 1.13.1
#18 414.9 C compiler for the host machine: cc (gcc 11.4.0 "cc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0")
#18 414.9 C linker for the host machine: cc ld.bfd 2.38
#18 414.9 C++ compiler for the host machine: c++ (gcc 11.4.0 "c++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0")
#18 414.9 C++ linker for the host machine: c++ ld.bfd 2.38
#18 414.9 Cython compiler for the host machine: cython (cython 3.0.11)
#18 414.9 Host machine cpu family: x86_64
#18 414.9 Host machine cpu: x86_64
#18 414.9 Program python found: YES (/usr/local/bin/pypy3)
#18 414.9 Run-time dependency python found: YES 3.9
#18 414.9 Program cython found: YES (/tmp/pip-build-env-v_vnvt3h/overlay/bin/cython)
#18 414.9 Compiler for C supports arguments -Wno-unused-but-set-variable: YES
#18 414.9 Compiler for C supports arguments -Wno-unused-function: YES
#18 414.9 Compiler for C supports arguments -Wno-conversion: YES
#18 414.9 Compiler for C supports arguments -Wno-misleading-indentation: YES
#18 414.9 Library m found: YES
#18 414.9
#18 414.9 ../meson.build:78:0: ERROR: Unknown compiler(s): [['gfortran'], ['flang'], ['nvfortran'], ['pgfortran'], ['ifort'], ['ifx'], ['g95']]
#18 414.9 The following exception(s) were encountered:
#18 414.9 Running `gfortran --version` gave "[Errno 2] No such file or directory: 'gfortran'"
#18 414.9 Running `gfortran -V` gave "[Errno 2] No such file or directory: 'gfortran'"
#18 414.9 Running `flang --version` gave "[Errno 2] No such file or directory: 'flang'"
#18 414.9 Running `flang -V` gave "[Errno 2] No such file or directory: 'flang'"
#18 414.9 Running `nvfortran --version` gave "[Errno 2] No such file or directory: 'nvfortran'"
#18 414.9 Running `nvfortran -V` gave "[Errno 2] No such file or directory: 'nvfortran'"
#18 414.9 Running `pgfortran --version` gave "[Errno 2] No such file or directory: 'pgfortran'"
#18 414.9 Running `pgfortran -V` gave "[Errno 2] No such file or directory: 'pgfortran'"
#18 414.9 Running `ifort --version` gave "[Errno 2] No such file or directory: 'ifort'"
#18 414.9 Running `ifort -V` gave "[Errno 2] No such file or directory: 'ifort'"
#18 414.9 Running `ifx --version` gave "[Errno 2] No such file or directory: 'ifx'"
#18 414.9 Running `ifx -V` gave "[Errno 2] No such file or directory: 'ifx'"
#18 414.9 Running `g95 --version` gave "[Errno 2] No such file or directory: 'g95'"
#18 414.9 Running `g95 -V` gave "[Errno 2] No such file or directory: 'g95'"
#18 414.9
#18 414.9 A full log can be found at /tmp/pip-install-y77ar9d0/scipy_1e543e0816ed4b26984[4155](https://github.com/zhengruifeng/spark/actions/runs/11357130578/job/31589506939#step:7:4161)33ae9079d/.mesonpy-xqfvs4ek/meson-logs/meson-log.txt
#18 414.9 [end of output]
```
see https://github.com/zhengruifeng/spark/actions/runs/11357130578/job/31589506939
### Was this patch authored or co-authored using generative AI tooling?
no
Closes #48489 from zhengruifeng/infra_scipy.
Authored-by: Ruifeng Zheng <ruifengz@apache.org>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>scipy
installation in dockerfile1 parent e92bf37 commit 224d3baCopy full SHA for 224d3ba
File tree
Expand file treeCollapse file tree
1 file changed
+1
-1
lines changedFilter options
- dev/infra
Expand file treeCollapse file tree
1 file changed
+1
-1
lines changed+1-1Lines changed: 1 addition & 1 deletion
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
152 | 152 |
| |
153 | 153 |
| |
154 | 154 |
| |
155 |
| - | |
| 155 | + | |
156 | 156 |
| |
157 | 157 |
|
0 commit comments