Request python package numpy version 1.19.5 for class 105 / ESPM 125

[Elias326]

Which package do you want installed? numpy version 1.19.5. For reference: (https://pypi.org/project/numpy/1.19.5/)

Please provide more information about the package, such as the version you require, missing functionality you need, and a URL to the website for the package.

Are there specific package dependencies? N/A

Which hub do you want it installed on? For example r.datahub, datahub, data100.datahub, etc. Datahub

Which class will use this package? IB 105 / ESPM 125 with Instructor Kathy Nagel

Include a link to appropriate entry in https://classes.berkeley.edu/ if available. If not, please mention class name & instructor.

Which semester will this package be used for? Fall 2021

This helps us clean up unused packages after a term ends.

Any additional information we should know about? When previously running the genomics library on this Notebook, we come across an error that states "ImportError: Numba needs NumPy 1.20 or less." I am also unsure if this will cause other conflicts with other packages so I want to make sure all packages run smoothly with appropriate versions. Thank you.

[felder]

@Elias326 when I open that notebook and look for "genomics" I see no import statements corresponding to that word.

Can you please go into more detail on what it means to run the genomics library on the notebook and what/where I need to click/type within the notebook in order to reproduce this error?

My thinking is maybe instead of downgrading numpy I could try upgrading numba. However, I need a reproducible test case.

[Elias326]

Hi @felder ! My bad for the misspelling. It's no genomics it's geonomics. The import is: import geonomics as gnx. There is a cell that imports all the packages. If you run that one, you can see the error:

import geonomics as gnx
import matplotlib.pyplot as plt
# these are for submissions
import otter
from otter.export import export_notebook 

# and also set Matplotlib's default plotting style and plot size
%matplotlib inline
plt.rcParams["figure.figsize"] = (9,4)

[felder]

@Elias326

Thanks! Looking for geonomics and running the cell with the import statement for it produces a suitable test case.

/opt/conda/lib/python3.9/site-packages/geopandas/_compat.py:106: UserWarning: The Shapely GEOS version (3.8.0-CAPI-1.13.1 ) is incompatible with the GEOS version PyGEOS was compiled with (3.9.1-CAPI-1.14.2). Conversions between both will be slow.
  warnings.warn(



NOTE: No display found. Using non-interactive Agg backend

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
/tmp/ipykernel_36/1860212330.py in <module>
----> 1 import geonomics as gnx
      2 import numpy as np
      3 import matplotlib.pyplot as plt
      4 
      5 # these are for submissions

/opt/conda/lib/python3.9/site-packages/geonomics/__init__.py in <module>
      1 # flake8: noqa
      2 
----> 3 from .main import *
      4 from . import ops
      5 from . import sim

/opt/conda/lib/python3.9/site-packages/geonomics/main.py in <module>
     39 """
     40 #geonomics imports
---> 41 from geonomics.sim.model import Model
     42 from geonomics.sim.params import (_read_params_file, _make_params_file,
     43                                   ParametersDict)

/opt/conda/lib/python3.9/site-packages/geonomics/sim/model.py in <module>
      9 
     10 #geonomics imports
---> 11 from geonomics.structs.landscape import _make_landscape
     12 from geonomics.structs.community import _make_community
     13 from geonomics.structs.genome import _make_genomic_architecture

/opt/conda/lib/python3.9/site-packages/geonomics/structs/__init__.py in <module>
----> 1 from . import landscape
      2 from . import genome
      3 from . import individual
      4 from . import species
      5 from . import community

/opt/conda/lib/python3.9/site-packages/geonomics/structs/landscape.py in <module>
      7 '''
      8 #geonomics imports
----> 9 from geonomics.utils.viz import _get_plt_lims, _plot_rasters, _check_display
     10 from geonomics.utils.io import _write_geotiff, _write_txt_array, _read_raster
     11 from geonomics.utils.spatial import _scale_raster, _make_nlmpy_raster

/opt/conda/lib/python3.9/site-packages/geonomics/utils/__init__.py in <module>
      1 from . import io
      2 from . import viz
----> 3 from . import spatial
      4 from . import _str_repr_

/opt/conda/lib/python3.9/site-packages/geonomics/utils/spatial.py in <module>
     22 
     23 try:
---> 24     from nlmpy import nlmpy
     25     with_nlmpy = True
     26 except ModuleNotFoundError:

/opt/conda/lib/python3.9/site-packages/nlmpy/nlmpy.py in <module>
     29 import numpy as np
     30 from scipy import ndimage
---> 31 from numba import jit
     32 
     33 #------------------------------------------------------------------------------

/opt/conda/lib/python3.9/site-packages/numba/__init__.py in <module>
    196 
    197 _ensure_llvm()
--> 198 _ensure_critical_deps()
    199 
    200 # we know llvmlite is working as the above tests passed, import it now as SVML

/opt/conda/lib/python3.9/site-packages/numba/__init__.py in _ensure_critical_deps()
    136         raise ImportError("Numba needs NumPy 1.17 or greater")
    137     elif numpy_version > (1, 20):
--> 138         raise ImportError("Numba needs NumPy 1.20 or less")
    139 
    140     try:

ImportError: Numba needs NumPy 1.20 or less

[Elias326]

Correct. This is the error I am seeing. I am hoping that if one package is updated that all packages in that cell work as well. Thank you!

[felder]

@Elias326 there are a few dependency issues here. In order to get this one cell to execute with the only warning being the display I had to install tskit, msprime, and pykrige as well. Additionally when I simply told pip to just install the packages I had other dependency conflicts.

I suspect the issue is related to the warning message pip gives me when I run it:

ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.

We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.

Namely the package maintainers may not have done the above which leads to pip not installing the right versions of each dependency.

In order to resolve all of the conflicts I have to run the following from a terminal in my own instance (restart your notebook kernel after doing so if you have the notebook open) and force these specific versions:

pip install numpy==1.20.3 numba==0.54.0 attrs==21.2.0 demes==0.1.2 tskit==0.3.7 msprime==1.0.2 pykrige==1.6.1

Fortunately I don't see any of those packages being specifically installed as part of the image build. It's certainly possible they represent dependencies for packages, however.

Before I make any changes @Elias326 can you run the above pip installation command in your own hub instance and then verify your notebooks all run correctly? Note that changes to your own hub are reverted each time your server is restarted. You can use this to your advantage to bring your environment back to the "default" state while testing.

If possible I'd like to make all of the necessary changes here at once.

[felder]

Elias would like to meet with me over zoom to work through this issue further. He says he is having other problems. Scheduled a zoom session for Thursday.

[yuvipanda]

I also worry that downgrading numpy might affect other classes. One option here is to add this command (copied from @felder above) at the top of the notebooks running into this issue:

!pip install numpy==1.20.3 numba==0.54.0 attrs==21.2.0 demes==0.1.2 tskit==0.3.7 msprime==1.0.2 pykrige==1.6.1

This should fix it just for that session, and not affect other classes.

numba/numba#7176 is the upstream issue fixing this in numba.

[felder]

Noting that I did meet with Elias last week to give him some tips and suggestions for how to go about tackling the issues he's experiencing with his notebook.

[ericvd-ucb]

turns out its a fiddly package that we need to get more info from the owner before we attempt to launch this notebook

[yuvipanda]

Any update here?

[balajialg]

@ericvd-ucb @Elias326 Any updates? Can we close this issue if it is not relevant anymore?

[balajialg]

@Elias326 @ericvd-ucb Closing this issue for now! Please feel free to reopen whenever the package is required.