Repository navigation
render-smoke test flakes under load: a 150ms budget has to cover PTY + Node startup #2177
Description
scripts/lib/render-smoke.test.mjs → "a paint landing just under the deadline is not failed by the render timer" fails intermittently under load, which fails npm run test:scripts and therefore validate and the whole local:gate.
✖ a paint landing just under the deadline is not failed by the render timer (403.360125ms)
AssertionError [ERR_ASSERTION]: expected pass, got: child did not render "MCP Servers" within 400ms
Hit twice in consecutive npm run local:gate runs on an unrelated branch; passes every time when npm run test:scripts is run on its own.
Cause
The test gives itself a 150ms budget that has to cover process startup, and startup alone nearly exhausts it:
const paintAt = 250;
const r = await runRenderSmoke({
...stub(`setTimeout(() => { console.log(MARKER); setInterval(() => {}, 1000); }, ${paintAt});`),
timeoutMs: paintAt + 150, // 400ms
surviveMs: 400,
});paintAt is measured from when the child's JS starts. renderTimer is armed in the parent, at spawn (render-smoke.mjs:324). So the 150ms has to absorb script(1) PTY allocation plus Node boot before the child's own 250ms timer even starts.
Measured on this machine, idle, 8 samples of spawn → first byte of child JS output through the same PTY wrapper:
66, 67, 68, 68, 68, 69, 71, 114 ms (max 114, budget 150)
So the margin is ~36ms at best when nothing else is running. Inside local:gate — where this runs alongside the rest of the suite — it goes over, the render timer fires before the paint, and the run is reported as "did not render", which is precisely the misdiagnosis the test exists to guard against.
Not a wrong assertion — a too-tight fixture
The behavior under test is right and worth keeping: the render deadline must be able to expire during the survival window without settling the run as a render failure. That shape only needs
actual_paint < timeoutMs < actual_paint + surviveMs
and the current constants satisfy it only when startup is under 150ms.
Proposed fix
Widen both constants so the same shape holds with real margin on both sides — e.g. timeoutMs: 1200, surviveMs: 1500 with paintAt: 250. Worst-case observed paint (~365ms) still lands well before 1200ms, and 1200ms still falls inside the survival window (which would end between ~1750ms and ~1900ms), so the timer still fires during it. Costs roughly a second of runtime on one test.
Do not fix it by removing the overlap — the overlap is the bug being guarded (see the comment above the test).
Done when
- The test does not depend on process startup fitting in a fixed sub-150ms budget
- The render deadline still expires inside the survival window, so the guard still guards
-
npm run test:scriptspasses repeatedly while the machine is loaded
Activity
fix(scripts): give the render-smoke overlap test room for process sta…
fix(scripts): give the render-smoke overlap test room for process sta…
test(scripts): anchor the render-smoke overlap paint to the parent's …