BUG: Building pandas via pip fails in AIX due to use of lower meson version

[KamathForAIX]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

If someone build master branch of pandas via pip in AIX, then they will receive the error as follows:

 pip3.9 install pandas -v

mdlist = compiler.get_command_to_archive_shlib()
    File "/tmp/pip-build-env-y55d21vm/overlay/lib/python3.9/site-packages/mesonbuild/compilers/compilers.py", line 948, in get_command_to_archive_shlib
      return self.linker.get_command_to_archive_shlib()
  AttributeError: 'NoneType' object has no attribute 'get_command_to_archive_shlib'
  The Meson build system
  Version: 1.2.1
  Source dir: /tmp
Found ninja-1.11.1 at /opt/freeware/bin/ninja

  Visual Studio environment is needed to run Ninja. It is recommended to use Meson wrapper:
  /tmp/pip-build-env-y55d21vm/overlay/bin/meson compile -C .

  ERROR: Unhandled python exception

      This is a Meson bug and should be reported!
  error: subprocess-exited-with-error
  
  Preparing metadata (pyproject.toml) did not run successfully.
  exit code: 2
  
  See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /opt/freeware/bin/python3.9 /opt/freeware/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpd79b_9px
  cwd: /tmp/pip-install-ihktll3n/pandas_3bab19dae760400388e20f1843d43267
  Preparing metadata (pyproject.toml) ... error
error: metadata-generation-failed

Encountered error while generating package metadata.

See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Issue Description

This is a Python exception that happened, which I have fixed here

Expected Behavior

pip3.9 install pandas should install pandas in AIX.

I would kindly request the community to increase the minimum required version to 1.6.0 if possible, and this will build Pandas in AIX and other targets as well.

I tested with making the change in pyproject.toml from "meson==1.2.1" to "meson==1.6.0" in AIX and it works.

Requesting the community to kindly consider this, and I will raise a pull request once you are okay.

[asishm]

Thanks for the report. I see that the python version you're trying to build against is 3.9 which has been removed in the main branch in #58238. Are you able to build pandas with python 3.10+ with meson 1.2.1?

[KamathForAIX]

Hi @asishm Nope. With 3.11 also, we hit the same issue.

[rhshadrach]

@KamathForAIX - can you repost the log with 3.11.

Given the log:

ERROR: Unhandled python exception

      This is a Meson bug and should be reported!

it looks like this should be reported to Meson.

[KamathForAIX]

@rhshadrach Please see the log below. Yes, this is indeed a meson bug. But it has been fixed and is available in meson versions > 1.3.0.

But Pandas still uses Meson 1.2.1 which blocks build in AIX. If we can, then I would like to raise a pull request with a change in pyproject.toml here, where we can update this to any version higher than 1.3.0 or (use the latest 1.6.0) and the corresponding meson python version as well (0.17).

# pip3.11 install pandas -v
Using pip 23.3.1 from /opt/freeware/lib/python3.11/site-packages/pip (python 3.11)
Collecting pandas
  Using cached pandas-2.2.3.tar.gz (4.4 MB)
  Running command pip subprocess to install build dependencies
  Collecting meson-python==0.13.1
    Using cached meson_python-0.13.1-py3-none-any.whl.metadata (4.1 kB)
  Collecting meson==1.2.1
    Using cached meson-1.2.1-py3-none-any.whl.metadata (1.7 kB)
  Collecting wheel
    Using cached wheel-0.45.1-py3-none-any.whl.metadata (2.3 kB)
  Collecting Cython~=3.0.5
    Using cached Cython-3.0.11-py2.py3-none-any.whl.metadata (3.2 kB)
  Collecting numpy>=2.0
    Using cached numpy-2.2.0.tar.gz (20.2 MB)
    Installing build dependencies: started
    Installing build dependencies: finished with status 'done'
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'done'
    Preparing metadata (pyproject.toml): started
    Preparing metadata (pyproject.toml): still running...
    Preparing metadata (pyproject.toml): still running...
    Preparing metadata (pyproject.toml): finished with status 'done'
  Collecting versioneer[toml]
    Using cached versioneer-0.29-py3-none-any.whl.metadata (16 kB)
  Collecting pyproject-metadata>=0.7.1 (from meson-python==0.13.1)
    Using cached pyproject_metadata-0.9.0-py3-none-any.whl.metadata (6.9 kB)
  Collecting packaging>=19.0 (from pyproject-metadata>=0.7.1->meson-python==0.13.1)
    Using cached packaging-24.2-py3-none-any.whl.metadata (3.2 kB)
  Using cached meson_python-0.13.1-py3-none-any.whl (79 kB)
  Using cached meson-1.2.1-py3-none-any.whl (962 kB)
  Using cached wheel-0.45.1-py3-none-any.whl (72 kB)
  Using cached Cython-3.0.11-py2.py3-none-any.whl (1.2 MB)
  Using cached pyproject_metadata-0.9.0-py3-none-any.whl (18 kB)
  Using cached versioneer-0.29-py3-none-any.whl (46 kB)
  Using cached packaging-24.2-py3-none-any.whl (65 kB)
  Building wheels for collected packages: numpy
    Building wheel for numpy (pyproject.toml): started
    Building wheel for numpy (pyproject.toml): finished with status 'done'
    Created wheel for numpy: filename=numpy-2.2.0-cp311-cp311-aix_7205_2419_64.whl size=7561349 sha256=f9b507c5e983c08b5be221ea71a96e13cf7d1cb78d32098107194fda605183f3
    Stored in directory: /.cache/pip/wheels/1c/e4/bd/59ab8366e3b2a18d5b6eed8127a78095e04682cc04746cecba
  Successfully built numpy
  Installing collected packages: wheel, versioneer, packaging, numpy, meson, Cython, pyproject-metadata, meson-python
  Successfully installed Cython-3.0.11 meson-1.2.1 meson-python-0.13.1 numpy-2.2.0 packaging-24.2 pyproject-metadata-0.9.0 versioneer-0.29 wheel-0.45.1
  WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
  Installing build dependencies ... done
  Running command Getting requirements to build wheel
  Getting requirements to build wheel ... done
  Running command Preparing metadata (pyproject.toml)
  + meson setup /tmp/pip-install-2s9520vq/pandas_dffbab11b7624a1596e4e1f26dcd7a73 /tmp/pip-install-2s9520vq/pandas_dffbab11b7624a1596e4e1f26dcd7a73/.mesonpy-k_7g307v/build -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --vsenv --native-file=/tmp/pip-install-2s9520vq/pandas_dffbab11b7624a1596e4e1f26dcd7a73/.mesonpy-k_7g307v/build/meson-python-native-file.ini
  Traceback (most recent call last):
    File "/tmp/pip-build-env-gx2w6m07/overlay/lib/python3.11/site-packages/mesonbuild/mesonmain.py", line 194, in run
      return options.run_func(options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-gx2w6m07/overlay/lib/python3.11/site-packages/mesonbuild/msetup.py", line 358, in run
      app.generate()
    File "/tmp/pip-build-env-gx2w6m07/overlay/lib/python3.11/site-packages/mesonbuild/msetup.py", line 183, in generate
      return self._generate(env, capture, vslite_ctx)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-gx2w6m07/overlay/lib/python3.11/site-packages/mesonbuild/msetup.py", line 253, in _generate
      captured_compile_args = intr.backend.generate(capture, vslite_ctx)
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-gx2w6m07/overlay/lib/python3.11/site-packages/mesonbuild/backend/ninjabackend.py", line 616, in generate
      self.generate_rules()
    File "/tmp/pip-build-env-gx2w6m07/overlay/lib/python3.11/site-packages/mesonbuild/backend/ninjabackend.py", line 1308, in generate_rules
      self.generate_dynamic_link_rules()
    File "/tmp/pip-build-env-gx2w6m07/overlay/lib/python3.11/site-packages/mesonbuild/backend/ninjabackend.py", line 2390, in generate_dynamic_link_rules
      cmdlist = compiler.get_command_to_archive_shlib()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-gx2w6m07/overlay/lib/python3.11/site-packages/mesonbuild/compilers/compilers.py", line 948, in get_command_to_archive_shlib
      return self.linker.get_command_to_archive_shlib()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  AttributeError: 'NoneType' object has no attribute 'get_command_to_archive_shlib'
  The Meson build system
  Version: 1.2.1
  Source dir: /tmp/pip-install-2s9520vq/pandas_dffbab11b7624a1596e4e1f26dcd7a73
  Build dir: /tmp/pip-install-2s9520vq/pandas_dffbab11b7624a1596e4e1f26dcd7a73/.mesonpy-k_7g307v/build
  Build type: native build
  Project name: pandas
  Project version: 2.2.3
  C compiler for the host machine: gcc -pthread (gcc 10.3.0 "gcc (GCC) 10.3.0")
  C linker for the host machine: gcc -pthread ld.aix 7.2.5
  C++ compiler for the host machine: g++ -pthread (gcc 10.3.0 "g++ (GCC) 10.3.0")
  C++ linker for the host machine: g++ -pthread ld.aix 7.2.5
  Cython compiler for the host machine: cython (cython 3.0.11)
  Host machine cpu family: ppc
  Host machine cpu: powerpc
  Program python found: YES (/opt/freeware/bin/python3.11)
  Found pkg-config: /opt/freeware/bin/pkg-config (0.29.2)
  Run-time dependency python found: YES 3.11
  Build targets in project: 53

  pandas 2.2.3

    User defined options
      Native files: /tmp/pip-install-2s9520vq/pandas_dffbab11b7624a1596e4e1f26dcd7a73/.mesonpy-k_7g307v/build/meson-python-native-file.ini
      buildtype   : release
      vsenv       : True
      b_ndebug    : if-release
      b_vscrt     : md

  Found ninja-1.11.1 at /opt/freeware/bin/ninja

  Visual Studio environment is needed to run Ninja. It is recommended to use Meson wrapper:
  /tmp/pip-build-env-gx2w6m07/overlay/bin/meson compile -C .

  ERROR: Unhandled python exception

      This is a Meson bug and should be reported!
  error: subprocess-exited-with-error
  
  Preparing metadata (pyproject.toml) did not run successfully.
  exit code: 2
  
  See above for output.

[KamathForAIX]

Upgrading pandas to use a higher version of meson in pyproject.toml fixes this issue and we will be able to build pandas via pip in AIX.

[rhshadrach]

Ah, I see - thanks! I misread on first go.

Assuming a PR is green on the CI, I think we'd be good to upgrade. cc @lithomas1 @mroeschke