build(edr_napi): migrate to napi-rs v3 - #1385
Conversation
🦋 Changeset detectedLatest commit: 0ea61b6 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
f51a652 to
64244fc
Compare
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #1385 +/- ##
==========================================
+ Coverage 79.43% 79.66% +0.22%
==========================================
Files 446 446
Lines 76644 76618 -26
Branches 76644 76618 -26
==========================================
+ Hits 60881 61036 +155
+ Misses 13636 13466 -170
+ Partials 2127 2116 -11 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
70a17b5 to
09d4d7f
Compare
09d4d7f to
ef469a5
Compare
The migration commit's `napi build --platform` invocation gets a
v3-side change in napi-rs CLI: v3 always passes `--target=<host-triple>`
to cargo, redirecting cdylib output and instrumentation profraws to
`target/<triple>/release/...` rather than the unsuffixed
`target/release/...`. v2 didn't do this.
`cargo llvm-cov show-env --sh` defaults to the unsuffixed dir, so the
TS-side coverage steps (which all run `pnpm test` between `show-env`
and `cargo llvm-cov report`, triggering `napi build` via pretest hooks)
end up with:
- `LLVM_PROFILE_FILE` pointing at `target/release/profraws/...`
- `napi build` running cargo with `--target`, which writes the
instrumented .node to `target/<triple>/release/`
- The .node loaded by mocha may not even be the freshly-instrumented
one, and any profraws it does emit land outside the path
`cargo llvm-cov report` reads.
Net effect: PR #1385's TS-side coverage upload reports 0% on every
napi-touched file, and the project's `Hits` line in the codecov diff
drops by ~5500 lines despite the .node being exercised normally by
the JS tests. Reported by the upstream review as a "real
instrumentation gap, not a reporting glitch."
Fix: detect the host triple via `rustc -vV` and thread it explicitly
through both `cargo llvm-cov show-env --sh --target=<triple>` and
`cargo llvm-cov report --target=<triple>`. Both calls now agree with
the `--target` napi-rs is implicitly adding, so paths line up and
profraws are picked up.
Three call sites, all matching the pattern:
- `edr-ci.yml` test-edr-ts step (`pnpm -C crates/edr_napi test`)
- `edr-ci.yml` Run integration tests step (the
`js/integration-tests/*` filter)
- `hardhat-tests.yml` Run hardhat-tests step
Verified via the cdylib output paths in the codecov upload logs:
- main (pre-PR): target/release/.fingerprint/...
- PR #1385: target/x86_64-unknown-linux-gnu/release/.fingerprint/...
Inheriting from rustc's host triple keeps the fix matrix-friendly:
ubuntu-24.04 -> x86_64-unknown-linux-gnu, macos-15 ->
aarch64-apple-darwin, windows-2025 -> x86_64-pc-windows-msvc.
The const-enum revert (commit 3089c19) re-added `--no-const-enum` to the napi build, which in v3 emits string enums (`CheatcodeErrorCode`, `TestStatus`, `MineOrdering`) as type-only unions instead of runtime enums. I caught the affected sites in `crates/edr_napi/test/*.ts` but missed three in `js/integration-tests/solidity-tests/test/unit.ts`: test/unit.ts(283,19): error TS2693: 'CheatcodeErrorCode' only refers to a type, but is being used as a value here. test/unit.ts(307,19): error TS2693: 'CheatcodeErrorCode' ... test/unit.ts(572,55): error TS2693: 'TestStatus' only refers to a type ... The CI's `Run integration tests` job fails at the `pnpm build:dev` pretest step (during `tsc --build` of the integration-tests package), which means the test step never runs and the codecov upload step is skipped — that's why PR #1385's patch coverage was still showing at 47% even after the cargo-llvm-cov `--target` fix landed: the broken job upload simply doesn't happen. Fix: replace each value access with a string literal cast to the type: CheatcodeErrorCode.UnsupportedCheatcode -> "UnsupportedCheatcode" as CheatcodeErrorCode CheatcodeErrorCode.MissingCheatcode -> "MissingCheatcode" as CheatcodeErrorCode TestStatus.Success -> "Success" as TestStatus Verified: `tsc --build --incremental .` clean in the integration-tests/solidity-tests package.
The napi-rs v3 migration commit was reconstructed from a base predating renovate #1520, which reverted the cargo-hack install pin in edr-ci.yml from v2.82.2 back to the stale v2.81.10. The three cargo-llvm-cov sites were consolidated into the setup-llvm-cov composite action (correctly pinned v2.82.2), leaving this inline install as the only regression.
Wodann
left a comment
There was a problem hiding this comment.
Some suggestions. I'm working on resolving them
Co-authored-by: Wodann <Wodann@users.noreply.github.com>
|
I addressed the most pressing comments I left. The other ones are nice-to-haves; so feel free to merge, if you're okay with the changes. |
|
/bench |
|
⏳ EDR CI for this commit hasn't passed yet, so the regression benchmark was not started. Comment |
|
/bench |
|
🚀 Starting regression benchmark for |
|
✅ Regression benchmark passed for |
I was able to remove |
|
Looks good to me! I'll rebase/merge in main to make sure we're not hitting any of the new ESLints, and then I'm happy to move forward. |
PR to migrate to napi-rs v3 with accompanying Hardhat 3 test PR: NomicFoundation/hardhat#8258
I also tested Hardhat 2 locally with Verdaccio and this works. I've left out the PR as it needs a good amount of fixes not relating to this PR to be able to match the newest EDR version.
I've left one migration behind the
compat-mode, as that might need some design.I've tried to keep APIs the same where possible, relying on a new napi-cli release to preserve const string enum behaviour.
Note that the ThreadsafeFunction teardown bug gets worse in Node 20, and a little bit in Node 22. I would suggest merging this AFTER migrating away from Node 20 (#1393), and to consider a reduced Node 22 test matrix or a test suite pass on failed exit given that we have now filed it as a known Hardhat 3 issue: NomicFoundation/hardhat#8322.Update (2026-07-02): the intermittent bindings crashes are now diagnosed and fixed in this PR. The known Node TSFN teardown bug was only the first of three stacked memory-safety bugs — the other two (a napi-rs custom-GC use-after-free, and JS-derived
napi::Errors dropped off the JS thread) persisted on every Node version. See "The flaky bindings crash: three stacked bugs" below. A first-chance-gdb stress harness on native arm64 went from ~7% crashes per suite run to 0/200 (musl) and 0/300 (gnu) with the two fix commits.Will follow-up if/when napi-rs/napi-rs#3370 is in. See napi-rs/napi-rs#3368.
Claude summary
Migrate
edr_napito napi-rs v3Migrates
edr_napiandedr_napi_corefrom napi-rs v2 to v3 (napi/napi-derive3.x,@napi-rs/cli^3.7.0), moving the bindings onto napi-rs's current major version line and its lifetime-typed API. The JS API surface is preserved except for two documented changes (see "Consumer-visible changes"); both Hardhat 2 and Hardhat 3 work without modification, apart from a 14-line test-fixture adjustment in Hardhat 3 (hardhat#8258).The first commit is the bulk mechanical migration; every later commit is a single focused concern (rebase adaptations, error propagation, tests, style). Reviewing commit-by-commit should work well.
Consumer-visible changes
The full list — everything else is byte-compatible at the API level:
TestResult.reason/.counterexample/.valueSnapshotGroupsare now class getters typedT | undefined(previously optional class fieldsreadonly reason?: T, runtime contract unchanged: absent →undefined). v3 emits everyTestResultmember as a getter, but only these three — formerly optional — change structural required-ness: TypeScript treats getters as required, so object literals satisfyingTestResultmust spell these keys out. The only consumer impact found (Hardhat 3 test fixtures).SuiteResultis now a plain object (interface) instead of a class — same field shapes, read identically. It is no longer a runtime value, though:module.exports.SuiteResultis dropped, sonew SuiteResult/instanceof SuiteResultstop working (Hardhat only reads fields).decodeConsoleLogInputsCallback/printLineCallbacknow surface as JSON-RPC internal-error responses on the triggering request, carrying the JS error message. Previously they were swallowed (print) or crashed the process (decode). This is the one intentional runtime behavior change in Rust code; covered by new regression tests.Not changed, by design: all 15 napi enums keep their runtime-enum shape (
--runtime-string-enum, new in@napi-rs/cli3.7.0 via napi-rs/napi-rs#3284, restores it for the 3 string enums that v3 would otherwise demote to type-only unions);MineOrdering.Fifo-style value access keeps working everywhere.What changed, by layer
Typed surface (bulk of the diff, mechanical)
JsFunction→Function<'env, Args, Ret>; the'envlifetime propagates through 7#[napi(object)]config structs and 4 napi-exposed methods (which now takeenv: &'env Envso the returnedObject<'env>can borrow from it).JsObject→Object<'env>for deferred-promise returns;JsUnknown→Unknown<'_>;Vec<JsString>→Vec<JsString<'env>>(the no-Debug/Display/Serializeprivate-key-leak guard is unchanged).Clonefrom JS-typed wrappers:#[derive(Clone)]dropped from ~25#[napi(object)]data carriers (verified none are cloned); manual deep-copyCloneimpls onFuzzCase/BaseCounterExampleviaUint8Array::with_data_copied.&str: FromNapiValue: 4pub kind: &'static strtag fields →String(round-trip pinned by a mocha test).#[napi(async_runtime)]on the 5 sync entry points that calltokio::runtime::Handle::current(), including theEdrContextconstructor — v3 no longer enters the tokio runtime context implicitly for sync#[napi]functions.ThreadsafeFunction call sites (6 + 1)
All rewritten to the v3 builder (
build_threadsafe_function().weak::<true>().build_callback(...)). Weak/strong semantics are preserved exactly: the 6 callbacks that v2unref(env)'d are weak; the test-suite-completion TSFN stays strong (keeps the process alive during a Solidity test run).ErrorStrategy::Fatalmaps to the v3 builder default (CalleeHandled = false). v3'scall_with_return_valuenow delivers the JS outcome as anapi::Result— the logger sites forward it through their result channels (consumer-visible change 3 above); the coverage/gas-report sites already did.Build & packaging
napi/napi-derive= 3 (lockfile: 3.10.0/3.5.7; 3.10.0 is a same-day release but carries the napi-rs#3357 memory-safety fix, so it's cooldown-allowlisted — see the bindings-crash section);napi-buildstays 2.3 (no v3 exists).dyn-symbolsre-enabled (a napi default that ourdefault-features = falsesilently dropped): N-API symbols resolve at runtime via libloading instead of link-time externs. Required forcargo testbinaries to link now thatedr_napi_corehas unit tests; also how every default-configured napi-rs package (and all MSVC builds) already works. Failure-mode nuance: a host missing a symbol fails at first call (stderr message + error status) instead of atdlopen— unreachable in practice fornapi8on Node ≥ 20.build_edr_napi.sh:--no-const-enumkept (const enums can't be value-imported underisolatedModules),--runtime-string-enumadded,-- --lockedseparator replaces--cargo-flags.prepublish.sh: v3 flag names, plus explicit--skip-optional-publish --no-gh-releaseso publish/release safety doesn't depend on call-site ordering or an absentGITHUB_TOKEN.napi.name→binaryName,triples→targets,napi universal→universalize.CI
cargo llvm-covjobs pinCARGO_BUILD_TARGET=$(rustc --print host-tuple)(3 jobs, identical comment): v3'snapi buildimplicitly passes--target=<host>, redirecting cdylib output totarget/<triple>/; without the pin, profraws and the.nodeland in different subtrees and napi-crate coverage reports 0%.Design decisions to weigh in on
ArrayBufferannotation withUint8Arrayruntime for the two Hardhat-2-facing callbacks (setCallOverrideCallback,decodeConsoleLogInputsCallback): the.d.tsmatches Hardhat 2's typings; producing a realArrayBufferis impossible (lifetime-carrying type vs'staticTSFN args).Buffer.from(x)accepts both. Documented at both sites; exercised end-to-end by mocha tests asserting on the actual runtime values.Either<T, ()>instead ofOption<T>on the threeTestResultgetters: napi-rs serializesOption::Noneasnull, but consumers test=== undefined;()serializes asundefined.SuiteResultPOJO /TestResultclass:SuiteResulthas no methods, and Hardhat reads three of its four fields multiple times per suite (testResults×3,id×3,warnings×2;durationNsis currently unread) — as a class, eachtestResultsread would re-clone the entireVec<TestResult>. The eager conversion is shallow:testResultsbecomes an array ofTestResultclass handles, so per-test state stays lazy.TestResultkeeps methods (stackTrace(),callTraces()) over un-serializable Rust state, so it stays a class with per-getter clones (same cost profile as v2's field codegen).compat-modefeature enabled: v3's typedObject<'_>doesn't implementToNapiValueby value, making it unusable as TSFN args (CallJsBackArgs: 'static); the deprecatedJsObjectis the remaining sanctioned way to build the subscription-event object inside a TSFN callback. Scoped to one module (#![allow(deprecated)]with rationale). Follow-up planned to remove it; first candidate is v3'sObjectRef(a'staticreference wrapper that implementsToNapiValueby value).Vec<JsString<'env>>kept forowned_accountsrather than aStringnewtype:JsStringisn't deprecated, the leak-prevention property is intact, and a newtype costs ~20 lines of manual trait impls plus an allocation per element for no gain.Suggested review focus
crates/edr_napi/src/logger.rs+crates/edr_napi_core/src/logger.rs— the error-propagation change (typedLoggerErrorvariants, fallibleDecodeConsoleLogInputsFn). The only deliberate runtime behavior change.crates/edr_napi/src/solidity_tests/test_results.rs— densest file: getter pattern, manualClonedeep-copies,kind: Stringtags.crates/edr_napi_core/src/subscription.rs+crates/edr_napi/src/subscription.rs—compat-mode/JsObjectusage and theArc-wrapped TSFN (v3ThreadsafeFunctionisn'tClone).index.d.tsdiff — fastest way to audit the consumer surface:export enum→export declare enum(cosmetic), theTestResult/SuiteResultchanges above, newSolidityStackTraceEntry/SolidityStackTracealiases (fixes a dangling type reference in the generated typings), no drops or renames.The flaky bindings crash: three stacked bugs (all fixed)
The intermittent bindings-test crashes on this PR (SIGSEGV / SIGTRAP /
Check failed: node->IsInUse(), arm64-clustered, scattered across Node versions) were three independent memory-safety bugs sharing one symptom pool. Diagnosed with a first-chance-gdb stress harness on native arm64 runners — post-mortem cores were useless, becausesignal-exitre-raises fatal signals viaprocess.killand destroys the faulting stack.napi_threadsafe_functionteardown data race (nodejs/node#55706) — fixed upstream by Mika Fischer (landedbff6ea49, Node 25+, backported to 24.13.1; never to the 20/22 LTS lines). Covered by the Node 24/26 matrix bump (build: add support for Node v24 (new default) & v26 #1507); user-facing tracker hardhat#8322. This was the originally-suspected (and only known) cause; it masked the two below, which kept crashing on Node 24/26.napi_refbehind everyBuffer/TypedArraydropped off the JS thread through an internal TSFN with a racy destroyed-flag check, and deletes refs the env already invalidated at teardown → V8 GlobalHandles corruption. EDR triggers this constantly (Uint8Arrays dropped on tokio/deallocator threads). Fixed upstream in napi 3.10.0; adopted here via lockfile bump + a cooldown-allowlist entry (same-day release, but it is the memory-safety fix).napi::Errordropped off the JS thread — anapi::Errorfrom a JS throw (TSFN callback) or promise rejection owns anapi_refto the JS error object, is (unsafely)Send, and itsDropdeletes that ref on whatever thread runs it. This PR's error bridges shipped such errors across channels and awaited rejections on the tokio runtime → the same GlobalHandles corruption; this was the dominant cause. v2 was immune only because throwing callbacks were swallowed or crashed the process, so the errors never crossed threads. Fixed in this PR: errors cross threads asStrings only — stringified in the TSFN result closures (JS thread), and vianapi_error::reason_and_forget(extract message +mem::forget, a bounded leak of one JS error object per failed callback) where rejections materialize on tokio threads. An upstream napi-rs issue will follow; the fix shape is routingError-ref deletion through their new per-env custom-GC handle, like buffers.Evidence (bindings suite looped on native arm64, per configuration):
napi::Errorfix (bug 3 fixed)One bad
napi_delete_referencecorrupts V8's GlobalHandles free-list, and the next handle operation crashes — wherever it happens to be. That's why crash sites wandered (mid-run insidecreateProvider, at-exit in the TSFN close sweep) and why each single-cause theory only lowered the rate instead of eliminating it.Verification
network-manager/edr(incl. JSON-RPC provider e2e),gas-analytics, andsolidity-test(incl. real EVM execution). Hardhat-side diff: hardhat#8258 (test fixtures only).Follow-ups (out of scope)
napi::Error's unsoundSendimpl (env-boundnapi_refdeleted inDropon whatever thread runs it — see bug 3 above); once fixed upstream,napi_error::reason_and_forgetcan be retired.compat-modeonce an alternative TSFN-args idiom is chosen (see design decision 4);ObjectRefis the first candidate to spike.napi build --cross-compileto replace the Docker-based Linux release legs.#[napi(module_exports)]instead ofnapi-build::setup()(cosmetic).