Cython is broken in conda build

[TernaryM01]

Steps To Reproduce

1. Install SageMath using conda/mamba: mamba create -n sagemath sage python=3.11.11
2. Launch Jupyter Lab using either jupyter lab or sage -n jupyterlab (while sage conda env is active).
3. Try to run any Cython code:

%%cython
from libc.math cimport cos, sin
cimport cython

@cython.boundscheck(False)
@cython.wraparound(False)
cpdef intersection(...):
  ...

Expected Behavior

Cython code gets compiled.

Actual Behavior

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
File ~/miniconda3/envs/sagemath/lib/python3.11/site-packages/sage/misc/cachefunc.pyx:1010, in sage.misc.cachefunc.CachedFunction.__call__ (build/cythonized/sage/misc/cachefunc.c:8371)()
   1009 try:
-> 1010     return self.cache[k]
   1011 except TypeError:  # k is not hashable

KeyError: ((), ())

During handling of the above exception, another exception occurred:

PackageNotFoundError                      Traceback (most recent call last)

. . .

PackageNotFoundError: gdlib not found

Additional Information

This bug has been reported by someone else here.

Environment

• OS: Ubuntu 22.04 (Windows WSL2)
• Sage Versions tried: 10.4, 10.3, 10.2, 9.8 (but older installation of 9.8 used to work)
• Python Versions tried: 3.12.8, 3.11.11, 3.10.16

Checklist

• I have searched the existing issues for a bug report that matches the one I want to file, without success.
• I have read the documentation and troubleshoot guide

[dimpase]

Seems to be that the issue is PackageNotFoundError: gdlib not found, no?

That is, try mamba install gdlib before jupyter lab

[TernaryM01]

I've dug deep into this issue, and the solution is not going to be that simple.
Running mamba install gdlib does nothing, because gdlib is already installed.

[dimpase]

maybe @tobiasdiez knows what's going on here

[tobiasdiez]

No real idea but there were problems with gdlib's pkconfig file (at least on windows).

Honestly, I think we should deprecate cython and fortan compilation. Sage is not really in the business to provide an IDE. Proper cython support (i.e here properly searching for gdlib) would require to call meson or a similar build system - numpy is indeed generating a meson file in the background and then call meson/ninja to build it for its fortan support. Except for very quick prototypes, just building your cython code with little meson file should give a better developer experience in my opinion.

[TernaryM01]

(Forgive me for GitHub's terrible UI about closing this issue. I have no intention to close this issue until Cython works again.)

I disagree about deprecating Cython and Fortran compilation. Sagemath is dedicated for scientific computing, and speed matters. I ran across this issue (of Cython compilation) when I was trying to run a sample Jupyter notebook from SageManifolds about rendering the picture of a black hole. It's quite computation-heavy, and you can see at cell number 38 is where Cython gets invoked the first time, to speed up a crucial rendering function by x10000. This very cool Sagemath showcase wouldn't be possible without Cython.

[tobiasdiez]

This is natively supported by cython: https://cython.readthedocs.io/en/latest/src/userguide/source_files_and_compilation.html#compiling-with-a-jupyter-notebook

The only advantage of Sage's %cython is that it tries to set the correct lib arguments etc for a couple of sage's dependencies. But as you noticed, this is only working half of the time.

[dimpase]

numpy is indeed generating a meson file in the background and then call meson/ninja to build it for its fortan support.

perhaps Sage can borrow this setup from numpy?

[user202729]

Maybe try %load_ext cython? This replaces the %%cython magic with the version provided by cython itself.

Just a band-aid at the moment, but better than not being able to use it at all.

[user202729]

Potentially related? #39275