FIX: Import importlib functions directly

[scott-huberty]

Hey @mscheltienne ,

I copied the import_optional_dependency code into an environment where I don't have the template-python deps installed, and ran into what I think is a bug in importlib. TL;DR in python 3.10.11+ I don't think you normally can do importlib.util.find_spec (see this comment from a Cpython developer).

I am not 100% sure about exactly what is going on, but here is what I've found out so far:

mamba create -n "importlib_test" python=3.10.13

# python

import importlib

importlib.util.find_spec  # raises error

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'importlib' has no attribute 'util'

But I can work around this with:

# python
from importlib.util import find_spec

# Now I can also do
import importlib
importlib.util.find_spec

So my current best guess is that one of the template-python dependencies does something like from importlib.util import ... which exposes the importlib.util module, allowing template-python to call it without issue?

---

Anyways, if your tests are passing on python 3.10.11+ then this might not be an actual issue. But if it is, then the PR here swaps out importlib.util.find_spec (which I actually really liked!) for a try....except block.

Up to you on whether you want to incorporate this to main.

[mscheltienne]

Interesting.. maybe from importlib.util import find_spec would be safer and avoid the try/except, I'll have to test it.

[scott-huberty]

Or actually now that I am thinking about it more...

We can just do

import importlib.util

....

if importlib.util.find_spec

So basically my updated understanding is that at least in python 3.10.11+, the supported API of importlib is to import submodules directly (i.e. don't rely on importlib to import submodules for you when doing import importlib)

EDIT:

I just saw your comment @mscheltienne , yes I think you are right!

[mscheltienne]

Yes, reading through the rest of that post, import importlib.util should be safe too.

[scott-huberty]

@mscheltienne WDYT? Is it weird to import importlib twice like this?

[mscheltienne]

I would replace it with 2 (very) explicit imports:

from importlib import import_module
from importlib.util import find_spec

(might have made a typo, I'm on my phone)

[mscheltienne]

Thanks @scott-huberty