Skip to content

[discussion] Use a rust crate to make astroid faster, or part of astroid faster #2014

Open

Description

I was reading this article lately. With pydantic, ruff and other python package using rust to provide faster python using binding offered by https://github.com/PyO3/pyo3 . I've been wondering if we should explore the possibility to do that too.

Potential contributor to astroid are scarce, so we rejected using cython earlier (In #606 (comment) and elsewhere with hippo91 but I can't find the link right now).

However:

  • rust is well loved and has been the most loved language for a few year on stackoverflow. So we might find contributors after all ?
  • ruff performance proves it's possible as a linter and also that it's a massive boost (10x or more) and a game changer
  • astroid's code is convoluted, maybe a way to avoid painfully typing it and maintaining it ? Daniel wanted to make it internal, well, porting to rust is a way to do whatever we like ? 😄

Problem I foresee:

  • I don't know how astroid works, it's not a given that we're going to architecture a new package better ?
  • We rely on the ast module right now, I don't know if we can rely on the ast from rust. The fact that ruff do not handle pattern matching seems to indicate that no we can't.

Maybe it's possible to use rust on the bottleneck only, i.e. on inference.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions