BUILD: pandas on conda main repo installs `numexpr`, while on `conda-forge` it does not

[Dr-Irv]

Installation check

• I have read the installation guide.

Platform

Windows-11-10.0.22631-SP0

Installation Method

conda install

pandas Version

2.2.2

Python Version

3.12 and 3.11

Installation Logs

When you do conda install pandas and it uses the main repo, numexpr is automatically installed, even though we document it as being optional:

conda install pandas
Channels:
 - defaults
 - gurobi
Platform: win-64
Collecting package metadata (repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: C:\Condadirs\envs\tnumexpr

  added / updated specs:
    - pandas


The following NEW packages will be INSTALLED:

  blas               pkgs/main/win-64::blas-1.0-mkl
  bottleneck         pkgs/main/win-64::bottleneck-1.3.7-py311hd7041d2_0
  intel-openmp       pkgs/main/win-64::intel-openmp-2023.1.0-h59b6b97_46320
  mkl                pkgs/main/win-64::mkl-2023.1.0-h6b88ed4_46358
  mkl-service        pkgs/main/win-64::mkl-service-2.4.0-py311h2bbff1b_1
  mkl_fft            pkgs/main/win-64::mkl_fft-1.3.10-py311h827c3e9_0
  mkl_random         pkgs/main/win-64::mkl_random-1.2.7-py311hea22821_0
  numexpr            pkgs/main/win-64::numexpr-2.8.7-py311h1fcbade_0
  numpy              pkgs/main/win-64::numpy-1.26.4-py311hdab7c0b_0
  numpy-base         pkgs/main/win-64::numpy-base-1.26.4-py311hd01c5d8_0
  pandas             pkgs/main/win-64::pandas-2.2.2-py311hea22821_0
  python-dateutil    pkgs/main/win-64::python-dateutil-2.9.0post0-py311haa95532
  python-tzdata      pkgs/main/noarch::python-tzdata-2023.3-pyhd3eb1b0_0
  pytz               pkgs/main/win-64::pytz-2024.1-py311haa95532_0
  six                pkgs/main/noarch::six-1.16.0-pyhd3eb1b0_1
  tbb                pkgs/main/win-64::tbb-2021.8.0-h59b6b97_0


Proceed ([y]/n)? n

When you install pandas from conda-forge, numexpr is not included:

conda install pandas -c conda-forge
Channels:
 - conda-forge
 - defaults
 - gurobi
Platform: win-64
Collecting package metadata (repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: C:\Condadirs\envs\tnumexpr

  added / updated specs:
    - pandas


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    intel-openmp-2024.2.1      |    h57928b3_1083         1.8 MB  conda-forge
    libblas-3.9.0              |     23_win64_mkl         5.0 MB  conda-forge
    libcblas-3.9.0             |     23_win64_mkl         5.0 MB  conda-forge
    libexpat-2.6.3             |       he0c23c2_0         136 KB  conda-forge
    libhwloc-2.11.1            |default_h8125262_1000         2.3 MB  conda-fore
    liblapack-3.9.0            |     23_win64_mkl         5.0 MB  conda-forge
    libsqlite-3.46.1           |       h2466b09_0         856 KB  conda-forge
    mkl-2024.1.0               |     h66d3029_694       104.3 MB  conda-forge
    numpy-2.1.1                |  py312h49bc9c5_0         6.7 MB  conda-forge
    pandas-2.2.2               |  py312h72972c8_1        13.5 MB  conda-forge
    pthreads-win32-2.9.1       |       hfa6e2cd_3         141 KB  conda-forge
    python-3.12.5              |h889d299_0_cpython        15.2 MB  conda-forge
    tbb-2021.13.0              |       hc790b64_0         148 KB  conda-forge
    ------------------------------------------------------------
                                           Total:       159.8 MB

The following NEW packages will be INSTALLED:

  intel-openmp       conda-forge/win-64::intel-openmp-2024.2.1-h57928b3_1083
  libblas            conda-forge/win-64::libblas-3.9.0-23_win64_mkl
  libcblas           conda-forge/win-64::libcblas-3.9.0-23_win64_mkl
  libexpat           conda-forge/win-64::libexpat-2.6.3-he0c23c2_0
  libhwloc           conda-forge/win-64::libhwloc-2.11.1-default_h8125262_1000
  libiconv           conda-forge/win-64::libiconv-1.17-hcfcfb64_2
  liblapack          conda-forge/win-64::liblapack-3.9.0-23_win64_mkl
  libsqlite          conda-forge/win-64::libsqlite-3.46.1-h2466b09_0
  libxml2            conda-forge/win-64::libxml2-2.12.7-h0f24e4e_4
  libzlib            conda-forge/win-64::libzlib-1.3.1-h2466b09_1
  mkl                conda-forge/win-64::mkl-2024.1.0-h66d3029_694
  numpy              conda-forge/win-64::numpy-2.1.1-py312h49bc9c5_0
  pandas             conda-forge/win-64::pandas-2.2.2-py312h72972c8_1
  pthreads-win32     conda-forge/win-64::pthreads-win32-2.9.1-hfa6e2cd_3
  python-dateutil    conda-forge/noarch::python-dateutil-2.9.0-pyhd8ed1ab_0
  python-tzdata      conda-forge/noarch::python-tzdata-2024.1-pyhd8ed1ab_0
  python_abi         conda-forge/win-64::python_abi-3.12-5_cp312
  pytz               conda-forge/noarch::pytz-2024.1-pyhd8ed1ab_0
  six                conda-forge/noarch::six-1.16.0-pyh6c4a22f_0
  tbb                conda-forge/win-64::tbb-2021.13.0-hc790b64_0
  ucrt               conda-forge/win-64::ucrt-10.0.22621.0-h57928b3_0
  vc14_runtime       conda-forge/win-64::vc14_runtime-14.40.33810-hcc2c482_20

Are we OK with this difference? @lithomas1

[mroeschke]

Since Anaconda controls the default channel, I suppose they have the right to include our "recommended" optional dependencies. IMO I think we are right not to include them through conda-forge as they are truly optional.

[Dr-Irv]

Since Anaconda controls the default channel, I suppose they have the right to include our "recommended" optional dependencies. IMO I think we are right not to include them through conda-forge as they are truly optional.

Well, yes, but do we need to document this?

I had some code that had installed pandas 2.2.2 from conda-forge, and got a new laptop, re-created the environment, which installed pandas from the default channel, and then I received all these numexpr warnings which didn't appear any more, so it was pretty confusing.

[lithomas1]

I think I agree with Matt here.

You can file an issue with anaconda here maybe if you want them to drop numexpr as a dep from their feedstock
https://github.com/AnacondaRecipes/pandas-feedstock/issues

[jorisvandenbossche]

I think it would be good to raise an issue about this in the AnacondaRecipes feedstock

[Dr-Irv]

The other option is that we change our documentation to reflect that installations from conda-forge (which we document) and installations from the main channel could be different.

[lithomas1]

The other option is that we change our documentation to reflect that installations from conda-forge (which we document) and installations from the main channel could be different.

Let's see what the Anaconda folks think first.
numexpr maintenance has been spotty a bit now, so it's possible we can convince them to drop the dep.