feat(shiki): use wasm engine

[avivkeller]

Using the WASM shiki on the server, and the JavaScript shiki on the client, should result in the best mix of speed and size for our use-case.

See nodejs/api-docs-tooling#320.

Rouhgly a 50% speed improvement on the server, per my testing.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated (UTC)
nodejs-org	✅ Ready (Inspect)	Visit Preview	Jun 26, 2025 7:34pm

[Copilot]

Pull Request Overview

This PR introduces a new approach to syntax highlighting by using a WASM-based engine on the server and a JavaScript-based engine on the web, optimizing for speed and bundle size.

• Minimal web highlighter now explicitly uses the JavaScript regex engine.
• Server highlighter is updated to initialize and use the WASM Oniguruma engine with top-level await.
• ESLint configuration is updated to support top-level await by using the latest ECMAScript version.

Reviewed Changes

Copilot reviewed 5 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
packages/rehype-shiki/src/minimal.mjs	Added explicit engine selection for the minimal (web) highlighter
packages/rehype-shiki/src/index.mjs	Integrated asynchronous WASM engine initialization for the server side
packages/rehype-shiki/src/highlighter.mjs	Removed the default engine creation to allow external engine injection
packages/rehype-shiki/package.json	Added dependency for the WASM Oniguruma engine
packages/rehype-shiki/eslint.config.js	Updated ECMAScript version to support top-level await

Files not reviewed (1)

• pnpm-lock.yaml: Language not supported

Comments suppressed due to low confidence (1)

packages/rehype-shiki/src/highlighter.mjs:17

• Since the default engine creation was removed, it may be helpful to document that an engine must now be provided via options when calling createHighlighter.

  const shiki = createHighlighterCoreSync({

[AugustinMauroy]

That's super !

[avivkeller]

@dario-piotrowicz https://github.com/nodejs/nodejs.org/actions/runs/15880572422/job/44779523238?pr=7896

Does OpenNext support WASM?

[dario-piotrowicz]

@dario-piotrowicz https://github.com/nodejs/nodejs.org/actions/runs/15880572422/job/44779523238?pr=7896

Does OpenNext support WASM?

sorry, not sure... this code comment suggests that it should work... I'll check with @vicb tomorrow, but I guess that we might be hitting some open-next bug here 🤔

[vicb]

@dario-piotrowicz https://github.com/nodejs/nodejs.org/actions/runs/15880572422/job/44779523238?pr=7896
Does OpenNext support WASM?

sorry, not sure... this code comment suggests that it should work... I'll check with @vicb tomorrow, but I guess that we might be hitting some open-next bug here 🤔

It looks like the code is trying to create the module from a UInt8Array:

That is not supported on Workers - they need the wasm to be loaded from a .wasm. Similar to why eval(...) doesn't work.

[ovflowd]

Can we store the Uint8Array as a file? The contents of it, or is there some way we can convert this to a wasm file. My understanding is we can pass the wasm instance to Shiki and we don't need to import this module....

[vicb]

Can we store the Uint8Array as a file? The contents of it, or is there some way we can convert this to a wasm file. My understanding is we can pass the wasm instance to Shiki and we don't need to import this module....

It should be a .wasm file.
I guess the Uint8Array is generated from some .wasm file so that file should be used instead - it doesn't look like it's part of @shikijs/engine-oniguruma. Maybe then a PR upstream.

[vicb]

Looking at the code it looks like the wasm file is in vscode-oniguruma - shiki uses v1.7.0

[avivkeller]

@vicb is there any way OpenNext can support loading WASM files via WebAssembly.instantiate-ing the content, like is done here?

 CompileError: WebAssembly.instantiate(): Wasm code generation disallowed by embedder

Shiki supplies custom imports (env and wasi_snapshot_preview1) via a custom importObject passed to WebAssembly.instantiate.

Hypothetically, we might be able to do some custom import resolving, but that's a lot of boilerplate. We could also conditionally use the JavaScript engine in OpenNext, and the WASM one otherwise? Does OpenNext have any specific environment variable we can use to identify when it's in use?

[codecov]

Codecov Report

Attention: Patch coverage is 11.76471% with 15 lines in your changes missing coverage. Please review.

Project coverage is 75.31%. Comparing base (9abc4fe) to head (a8899cf).
Report is 5 commits behind head on main.

✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
packages/rehype-shiki/src/index.mjs	11.76%	15 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7896      +/-   ##
==========================================
- Coverage   75.45%   75.31%   -0.14%     
==========================================
  Files         101      101              
  Lines        8311     8326      +15     
  Branches      218      218              
==========================================
  Hits         6271     6271              
- Misses       2038     2053      +15     
  Partials        2        2              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[avivkeller]

Ah, I see cloudflare/cloudflare-docs#13469 documents that this is not allowed. Is there a way to enable it on our end, knowing the security concerns?

If not, I think the conditional JS Engine is the way to go.

[vicb]

@vicb is there any way OpenNext can support loading WASM files via WebAssembly.instantiate-ing the content, like is done here?

Yes you can WebAssembly.instantiate(module) (a .wasm file uploaded along your source code), just not WebAssembly.instantiate(buffer)

Ah, I see cloudflare/cloudflare-docs#13469 documents that this is not allowed. Is there a way to enable it on our end, knowing the security concerns?

The security concerns are for Cloudflare too - that's why there is no way to instantiate from a buffer.
But instantiating from a module should work.
Using the JS engine is a good solution as well.

[avivkeller]

Thank you for all your help! I'll investigate this!

[vicb]

Thank you for all your help! I'll investigate this!

I pointed to the .wasm file to use in an earlier comment

[avivkeller]

In case there isn't a viable solution (which, hopefully, it seems there is), is there a way to tell when our code in running in Cloudflare, versus everything else, so we can go with the conditional JS engine?

[vicb]

In case there isn't a viable solution (which, hopefully, it seems there is), is there a way to tell when our code in running in Cloudflare, versus everything else, so we can go with the conditional JS engine?

You can use "Cloudflare" in globalThis for testing purpose as a runtime check.
If you go down that road, we could define process.env.OPEN_NEXT_CLOUDFLARE=1 so that the code can be tree shaken during the build.

[avivkeller]

Thank you again for your help! We will, unfortunately, need to conditionally use the JavaScript RegExp engine when we are on Cloudflare Workers.

If we were to import the WASM file like:

// Also available at "vscode-oniguruma/release/onig.wasm"
import wasm from "shiki/onig.wasm";
await import("shiki/onig.wasm");

It would result in errors like, since the imports need to be defined before we import the WASM file:

Module not found: Can't resolve 'wasi_snapshot_preview1'
Module not found: Can't resolve 'env'

[ovflowd]

LGTM!

[github-actions]

Lighthouse Results

URL	Performance	Accessibility	Best Practices	SEO	Report
/en	🟢 97	🟢 100	🟢 100	🟢 91	🔗
/en/about	🟢 100	🟢 96	🟢 100	🟠 82	🔗
/en/about/previous-releases	🟢 99	🟢 96	🟢 100	🟠 83	🔗
/en/download	🟢 98	🟢 100	🟢 100	🟢 91	🔗
/en/blog	🟢 100	🟢 100	🟢 96	🟢 92	🔗