Skip to content

fix(bench): build library packages before bundle-size in CI, fail loudly - #30

Merged
koficodedat merged 1 commit into
mainfrom
fix/bench-bundle-size-missing-build-step
Jul 27, 2026
Merged

koficodedat merged 1 commit into
mainfrom
fix/bench-bundle-size-missing-build-step

Conversation

@koficodedat

Copy link
Copy Markdown
Contributor

Summary

The live docs page (/benchmarks/) shows ERROR for neutro/form and neutro/form (minimal) Gzip size. This is the same bug as an earlier fix (5161336, "fix(bench): resolve bundle-size ERROR + scorecard self-comparison bug") resurfacing, not a new bug.

Root cause: bench-full.yml and bench-weekly.yml never run pnpm build before the bundle-size step. The bundle-size fixtures (bench/fixtures/bundle/neutro.ts, neutro-minimal.ts) import @neutro/form-core and @neutro/form (the alias package) by name — resolved via each package's real dist/ output (gitignored). Unlike bench:core (Vitest, aliases straight to src/) or the demo apps (same alias trick in their Vite configs), esbuild resolves these through real package resolution, so a fresh CI checkout with no dist/ fails to bundle them. measure.ts's try/catch silently recorded status: "error" instead of throwing, so the job reported success while publishing a broken result straight to the live docs page.

The earlier fix only rebuilt packages and regenerated the docs by hand, once, locally — it never touched the workflows, so it recurred the very next time bench-full.yml actually ran (the v0.5.1 tag push a few hours ago).

Changes

  • Added a root pnpm install --frozen-lockfile + pnpm build step to both bench-full.yml and bench-weekly.yml, before bench's own --ignore-workspace installs — matches the pattern ci.yml's bench-apps-typecheck job already uses.
  • bench/suites/bundle/measure.ts now calls process.exit(1) if any neutro/form* entry comes back status: "error" — a failure to bundle our own library is an environment bug, not a real result, and must never be silently published again.
  • Regenerated bench/results/baseline.json and docs/benchmarks/index.md from a full local bench:full run (packages built first) to fix the currently-live broken page immediately: neutro/form is now 11.9 KB gzip, neutro/form (minimal) is 8.0 KB.

Test plan

  • Verified the bug reproduces: reverted the fix locally, confirmed tsc/build-less bundle-size run produces Could not resolve "@neutro/form-core" matching the committed baseline.json's recorded error
  • Ran the full local pipeline (bench:apps:build, bench:core, bench:correctness, bench:browser, bench:bundle-size, bench:merge, copy latest.json → baseline.json, bench:generate) with packages built first — all neutro/form* bundle-size entries now status: "ok"
  • pnpm lint && pnpm exec tsc --noEmit && pnpm build && pnpm test all pass (269 test files, 10838 tests)
  • Confirmed my new code in measure.ts doesn't introduce any new tsc errors beyond bench's pre-existing (unrelated, ungated) missing-@types/node gap

Done in an isolated worktree per request; ready to merge to main once reviewed.

🤖 Generated with Claude Code

https://claude.ai/code/session_014iNaPqgsuTJYYDw4Py24Dc

bench-full.yml and bench-weekly.yml never ran `pnpm build` before the
bundle-size step. The bundle-size fixtures import @neutro/form-core
and @neutro/form (the alias package) by name, resolved via each
package's real dist/ output (gitignored) -- unlike bench:core (which
aliases straight to src/ in vitest.config.ts) or the demo apps (same
alias trick in their vite configs). A fresh CI checkout has no dist/
until something builds it, so esbuild failed to resolve both imports,
measure.ts's try/catch quietly recorded status: "error" instead of
throwing, and that got merged, baselined, and published straight to
the live docs page as a Gzip size of "ERROR".

This is the same bug as the earlier bundle-size fix (5161336) --
that fix rebuilt packages and regenerated the docs by hand once, but
never added a build step to the workflows, so it recurred the very
next time bench-full.yml ran for real (the v0.5.1 tag push).

Fixes:
- Added a root `pnpm install --frozen-lockfile` + `pnpm build` step to
  both workflows, before bench's own --ignore-workspace installs,
  matching the pattern ci.yml's bench-apps-typecheck job already uses.
- measure.ts now fails the job (process.exit(1)) if any neutro/form*
  entry comes back with status "error" -- a failure to bundle our OWN
  library is an environment bug, not a real result, and must never be
  silently published as an ERROR badge again.
- Regenerated bench/results/baseline.json and docs/benchmarks/index.md
  from a full local bench:full run (packages built first) to fix the
  currently-live broken page immediately rather than waiting for the
  next scheduled/tagged run: neutro/form is now 11.9 KB gzip, neutro/form
  (minimal) is 8.0 KB, both correctly measured.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iNaPqgsuTJYYDw4Py24Dc
@koficodedat
koficodedat merged commit df0c1dd into main Jul 27, 2026
3 checks passed
@koficodedat
koficodedat deleted the fix/bench-bundle-size-missing-build-step branch July 27, 2026 12:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant