silence import resolution errors from an existing compiled extension module

[Stamppot82]

Summary

Environment: ty v0.0.1a12, Python 3.12, macOS, uv project

Problem: ty check reports unresolved-import errors for Cython extensions compiled to .so files, despite the imports working correctly at runtime.

Console Output:

error[unresolved-import]: Cannot resolve imported module `.math_utils`
 --> src/__init__.py:5:7
  |
5 | from .math_utils import add
  |       ^^^^^^^^^^

Project Structure:

src/
├── __init__.py
├── math_utils.pyx                    # Cython source
└── math_utils.cpython-312-darwin.so  # Compiled extension (exists)

Issue: ty only looks for .py files but not compiled .so extensions. The import from .math_utils import ... fails because ty cannot find a [math_utils.py]

Expected: ty should resolve imports from compiled Cython extensions when:

A .pyx source file exists
A corresponding .so file is present
The package is properly built/installed

Impact:
False positives make ty unusable for projects using Cython extensions, which are common in scientific Python, performance-critical libraries, and hybrid codebases.

Workaround: None effective (stub files don't work for dynamic imports from compiled extensions).

[AlexWaygood]

We don't plan to add support for inferring types from Cython .pyx sources in the near future, I'm afraid. Just to be able to read that source code (let alone infer types), we'd either have to make major changes to our existing parser or write a new parser.

Workaround: None effective (stub files don't work for dynamic imports from compiled extensions).

Unfortunately I believe stub files are the only solution to this problem for now, both for ty and any other Python type checker

[carljm]

I think it's very unlikely that we ever attempt to parse .pyx files, unless it becomes a specified typing feature that all Python type checkers support (I think zero type checkers support it today).

stub files don't work for dynamic imports from compiled extensions

Can you say more about this? What's an example of a "dynamic import" that would be supported by parsing pyx files but can't be supported in a stub file?

[AlexWaygood]

see astral-sh/ruff#10250 for the equivalent Ruff issue, which would be a pre-requisite for this even being a possibility

[erictraut]

If I understand the bug report correctly, the OP isn't expecting ty to parse the extension module or infer precise types for symbols imported from it. Any such symbols can be treated as Unknown if a stub file isn't present.

The question is whether ty should report an import resolution error here. Other type checkers (including mypy, pyright, and pyrefly) do not report an import resolution error if the import statement targets an extension module.

[carljm]

Yes, I think silencing import errors in that case makes sense. I think it should be based on the existence of a .so file in the import path; .pyx files shouldn't be considered.

[erictraut]

FWIW, pyright looks for the following file extensions to identify extension module binaries.

const supportedNativeLibExtensions = ['.pyd', '.so', '.dylib'];

One additional gotcha ... when trying to resolve module foo, it's not enough to simply look for foo.so, etc. because extension module file names can include compound file extensions that encode additional information like platform details. For example: foo.cpython-32m.so.

[carljm]

I'm curious why you found it important to include .pyd files? AFAIK the runtime doesn't ever import from those. Are there cases where you are unable to find a .so or .dylib but you are able to find a .pyd?

[JelleZijlstra]

At runtime Python accepts .so files with the suffixes in importlib.machinery.EXTENSION_SUFFIXES:

>>> import importlib.machinery
>>> print(importlib.machinery.EXTENSION_SUFFIXES)
['.cpython-312-darwin.so', '.abi3.so', '.so']

So to be strictly in line with the runtime, type checkers should only allow extension modules with these suffixes. The values are machine and version dependent; from looking at the code .dll is allowed under Cygwin, for example.

[erictraut]

I'm curious why you found it important to include .pyd files? AFAIK the runtime doesn't ever import from those.

My understanding is that the runtime does import ".pyd" files on Windows. I did a quick internet search and asked ChatGPT, and they seem to agree. I don't have a Windows machine handy at the moment though, so I can't confirm myself.

[JelleZijlstra]

I looked at the CPython code and that seems right; grep for PYD_TAGGED_SUFFIX.