ImportError: symbol not found in flat namespace '_bcf_float_missing' on Apple Silicon (osx-arm64)

[michelesurano]

Hi,

Thank you for developing and maintaining MetaXcan, it's a great tool!

I am writing to report an error I encountered when trying to run Predict.py on a Mac with Apple Silicon (M-series chip, osx-arm64) using the imlabtools conda environment created from the updated conda_env.yaml.

SYSTEM

• macOS (Apple Silicon, osx-arm64)
• Python 3.9
• conda environment created from software/conda_env.yaml
• cyvcf2==0.30.28 installed via pip (as specified in the yaml)

STEPS TO REPRODUCE

1. Clone the repository
2. Create the conda environment: conda env create -f software/conda_env.yaml
3. Activate the environment: conda activate imlabtools
4. Run python3 Predict.py with a VCF file

The environment is created successfully, but running Predict.py gives a numpy/scipy incompatibility error:

UserWarning: A NumPy version >=1.22.4 and <1.29.0 is required for this version of SciPy (detected version 2.0.2)  import scipy.stats as stats
Traceback (most recent call last):
  File "/Users/michele/MetaXcan/software/Predict.py", line 16, in <module>
    from metax.genotype import Genotype
  File "/Users/michele/MetaXcan/software/metax/genotype/Genotype.py", line 1, in <module>
    from ..misc import GWASAndModels
  File "/Users/michele/MetaXcan/software/metax/misc/GWASAndModels.py", line 7, in <module>
    from ..gwas import Utilities as GWASUtilities
  File "/Users/michele/MetaXcan/software/metax/gwas/Utilities.py", line 7, in <module>
    from . import GWAS
  File "/Users/michele/MetaXcan/software/metax/gwas/GWAS.py", line 5, in <module>
    import scipy.stats as stats
  File "/Users/michele/miniconda3/envs/imlabtools/lib/python3.9/site-packages/scipy/stats/__init__.py", line 605, in <module>
    from ._stats_py import *
  File "/Users/michele/miniconda3/envs/imlabtools/lib/python3.9/site-packages/scipy/stats/_stats_py.py", line 37, in <module>
    from scipy.spatial.distance import cdist
  File "/Users/michele/miniconda3/envs/imlabtools/lib/python3.9/site-packages/scipy/spatial/__init__.py", line 110, in <module>
    from ._kdtree import *
  File "/Users/michele/miniconda3/envs/imlabtools/lib/python3.9/site-packages/scipy/spatial/_kdtree.py", line 4, in <module>
    from ._ckdtree import cKDTree, cKDTreeNode
  File "_ckdtree.pyx", line 1, in init scipy.spatial._ckdtree
ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject

I've looked for this error message and found here (https://stackoverflow.com/questions/78634235/numpy-dtype-size-changed-may-indicate-binary-incompatibility-expected-96-from) a workaround. They suggest to pin a numpy version <= 2.0, specifically to run numpy==1.26.4 because pip overwrites numpy 1.26 (specified in the yaml) with numpy 2.0.2 when installing cyvcf2==0.30.28, since cyvcf2 requires numpy>=2.0.

However, after manually pinning numpy back to 1.26.4 with pip install numpy==1.26.4, a second error appears:

Traceback (most recent call last):
  File "/Users/michele/MetaXcan/software/Predict.py", line 272, in <module>
    run(args)
  File "/Users/michele/MetaXcan/software/Predict.py", line 160, in run
    samples = load_samples(args)
  File "/Users/michele/MetaXcan/software/Predict.py", line 87, in load_samples
    from metax.genotype import CYVCF2Genotype
  File "/Users/michele/MetaXcan/software/metax/genotype/CYVCF2Genotype.py", line 2, in <module>
    from cyvcf2 import VCF
  File "/Users/michele/miniconda3/envs/imlabtools/lib/python3.9/site-packages/cyvcf2/__init__.py", line 1, in <module>
    from .cyvcf2 import (VCF, Variant, Writer, r_ as r_unphased, par_relatedness,
ImportError: dlopen(/Users/michele/miniconda3/envs/imlabtools/lib/python3.9/site-packages/cyvcf2/cyvcf2.cpython-39-darwin.so, 0x0002): symbol not found in flat namespace '_bcf_float_missing'

How can I solve this issue please? Any help would be highly appreciated, thanks in advance!

[Fnyasimi]

This looks like a software dependency issues, I would suggest installing the tools one by one to find a good combination that works for the mac m series, you can also try the old spec conda version maybe it will work