Repository navigation
fix(bench): build library packages before bundle-size in CI, fail loudly - #30
Merged
Merged
Conversation
bench-full.yml and bench-weekly.yml never ran `pnpm build` before the bundle-size step. The bundle-size fixtures import @neutro/form-core and @neutro/form (the alias package) by name, resolved via each package's real dist/ output (gitignored) -- unlike bench:core (which aliases straight to src/ in vitest.config.ts) or the demo apps (same alias trick in their vite configs). A fresh CI checkout has no dist/ until something builds it, so esbuild failed to resolve both imports, measure.ts's try/catch quietly recorded status: "error" instead of throwing, and that got merged, baselined, and published straight to the live docs page as a Gzip size of "ERROR". This is the same bug as the earlier bundle-size fix (5161336) -- that fix rebuilt packages and regenerated the docs by hand once, but never added a build step to the workflows, so it recurred the very next time bench-full.yml ran for real (the v0.5.1 tag push). Fixes: - Added a root `pnpm install --frozen-lockfile` + `pnpm build` step to both workflows, before bench's own --ignore-workspace installs, matching the pattern ci.yml's bench-apps-typecheck job already uses. - measure.ts now fails the job (process.exit(1)) if any neutro/form* entry comes back with status "error" -- a failure to bundle our OWN library is an environment bug, not a real result, and must never be silently published as an ERROR badge again. - Regenerated bench/results/baseline.json and docs/benchmarks/index.md from a full local bench:full run (packages built first) to fix the currently-live broken page immediately rather than waiting for the next scheduled/tagged run: neutro/form is now 11.9 KB gzip, neutro/form (minimal) is 8.0 KB, both correctly measured. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014iNaPqgsuTJYYDw4Py24Dc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The live docs page (
/benchmarks/) showsERRORforneutro/formandneutro/form (minimal)Gzip size. This is the same bug as an earlier fix (5161336, "fix(bench): resolve bundle-size ERROR + scorecard self-comparison bug") resurfacing, not a new bug.Root cause:
bench-full.ymlandbench-weekly.ymlnever runpnpm buildbefore the bundle-size step. The bundle-size fixtures (bench/fixtures/bundle/neutro.ts,neutro-minimal.ts) import@neutro/form-coreand@neutro/form(the alias package) by name — resolved via each package's realdist/output (gitignored). Unlikebench:core(Vitest, aliases straight tosrc/) or the demo apps (same alias trick in their Vite configs), esbuild resolves these through real package resolution, so a fresh CI checkout with nodist/fails to bundle them.measure.ts's try/catch silently recordedstatus: "error"instead of throwing, so the job reported success while publishing a broken result straight to the live docs page.The earlier fix only rebuilt packages and regenerated the docs by hand, once, locally — it never touched the workflows, so it recurred the very next time
bench-full.ymlactually ran (the v0.5.1 tag push a few hours ago).Changes
pnpm install --frozen-lockfile+pnpm buildstep to bothbench-full.ymlandbench-weekly.yml, before bench's own--ignore-workspaceinstalls — matches the patternci.yml'sbench-apps-typecheckjob already uses.bench/suites/bundle/measure.tsnow callsprocess.exit(1)if anyneutro/form*entry comes backstatus: "error"— a failure to bundle our own library is an environment bug, not a real result, and must never be silently published again.bench/results/baseline.jsonanddocs/benchmarks/index.mdfrom a full localbench:fullrun (packages built first) to fix the currently-live broken page immediately:neutro/formis now 11.9 KB gzip,neutro/form (minimal)is 8.0 KB.Test plan
tsc/build-less bundle-size run producesCould not resolve "@neutro/form-core"matching the committedbaseline.json's recorded errorbench:apps:build,bench:core,bench:correctness,bench:browser,bench:bundle-size,bench:merge, copylatest.json→baseline.json,bench:generate) with packages built first — allneutro/form*bundle-size entries nowstatus: "ok"pnpm lint && pnpm exec tsc --noEmit && pnpm build && pnpm testall pass (269 test files, 10838 tests)measure.tsdoesn't introduce any newtscerrors beyond bench's pre-existing (unrelated, ungated) missing-@types/nodegapDone in an isolated worktree per request; ready to merge to
mainonce reviewed.🤖 Generated with Claude Code
https://claude.ai/code/session_014iNaPqgsuTJYYDw4Py24Dc