[PROPOSAL] Integrate Napi.rs to TSC to gradually migrate to Rust 🦀

[markmssd]

This PR is to contribute a suggestion, showcasing a working code example. It is not meant to be merged.

The web development ecosystem has evolved a lot, moving towards faster technologies, usually Rust or Golang. For example, Turbopack, Rspack, swc or esbuild (already used in this repo).

Unfortunately, those projects can only compile Typescript, not type check. Type checking remains a problem in large projects due to its performance.

There have been attempts at rewriting the TSC type checker in Rust, mainly by the well known @kdy1, creator of SWC, with the https://github.com/dudykr/stc project. However, keeping up with the Typescript changes is no easy task.

For this proposal, I use NAPI-RS directly in the Typescript repo, allowing us to gradually migrate the codebase to Rust. This is of course easier said than done, but I believe if the hot paths were migrated, it could already give a great performance boost.

This example is fully working, as you can see all the checks passing. I have only migrated 2 methods, escapeLeadingUnderscores and unescapeLeadingUnderscores, for the sake of demonstration. They are implemented in lib.rs, and running npm run build:napi will create rs.js and rs.d.ts files, allowing JS code to call them directly.

I'd love to hear your thoughts with such a solution. If there is interest, I can open another PR which would integrate it properly (using Hereby) and have all CI workflows passing.

PS: 2 CI workflows are failing, mostly due to configuration that needs to be tweaked, but npm test passes successfully for Linux, Windows, and MacOS.

[markmssd]

@microsoft-github-policy-service agree

[asukaminato0721]

infamous: well known for some bad quality.

I guess you want to say: not famous?

[MartinJohns]

This would prevent dogfooding.

[RyanCavanaugh]

This is of course easier said than done, but I believe if the hot paths were migrated, it could already give a great performance boost.

We've investigated this multiple times and it doesn't really work as you'd hope. The overhead from Napi.rs <-> JS is still higher than a monomorphic JS <-> JS call, and there is no singular "hot path" you can optimize (most time is spent in the checker where the call graph can and does go anywhere in checker.ts). Believe me, if you could get perf gains out of this, we'd have done it already.

[markmssd]

Thanks for the clear explanation @RyanCavanaugh ! That's exactly what I was afraid of, but I was hoping that the Rust performance boost would compensate for the overhead. For example, I see in the NAPI-RS docs that using JsBuffer:

Passing data between JavaScript and Rust using JsBuffer has a small overhead so you might prefer it over other types.

I didn't compare it in my PoC though. I might try to convert more util functions to Rust and try it out when I get more time. I'm going to close the PR now, but at least the working code remains if it's ever reconsidered in the future!