ci(judge-e2e): skip the engine on JS-only changes and cut its build from ~37 min

[andersonleal]

Why

Judge E2E took ~38 min per run, 37.1 of it in the real-engine step, while its 11 cases take ~27 s. The rest was compilation (run 37926568171):

Suite	Build	Tests
judge-semif / decider / laya / clef	7m38s / 7m31s / 8m24s / 7m58s	~2 s each
typesafe / openai / judge-provider / judge	~1m10s–1m20s each	2–5 s each

• llama.cpp was built from scratch four times, in the shipping configuration: every CPU variant plus the Vulkan shaders. The tests run tiny models on CPU.
• Eight separate Cargo workspaces each rebuilt the same dependencies.
• rust-cache never hit. The workflow runs only on pull_request, so caches are PR-scoped and no other PR can read them (chore: clean up remaining iii-stream and pubsub references (MOT-3619) #1353 got "No cache found" while refactor(judge-provider,judge-typesafe,judge-clef,judge-decider,judge-semif,judge-laya): MOT-5333 follow-ups (MOT-5342) #1356's 1 GB cache existed). semif/decider were not in its list, and it purges path crates, llama.cpp included.

The job also ran in full for PRs that touch only packages/console-ui (types only) or the root pnpm files, which the engine suite never exercises. #1353 paid 38 min for one .d.mts. Those paths changed 54 times in the last month.

What

• Skip the engine on JS-only changes: a scope step diffs the PR merge commit against its first parent. The Rust/engine steps run only when an engine-relevant path changed (on.pull_request.paths minus the four JS-only entries). The UI Vitest and tsc builds still run on every trigger.
• crates/llama-native: III_LLAMA_CPP_MINIMAL=1 builds one CPU module and no Vulkan. It is meant for test builds only; published builds are unchanged.
• One CARGO_TARGET_DIR for all eight suites, so each shared unit builds once. judge-laya's opt-level = 3 still gets its own llama.cpp. copy_into already removes before copying, so the shared profile dir is safe.
• Removed rust-cache, which never helped and stored 1 GB in the repo cache quota per PR, and the Vulkan apt step. The hosted image already ships cmake, curl, tar and patch.

Verification

• Local run of the engine step with these settings: 11/11 pass in 245 s on 32 cores. llama.cpp is built twice, each with only libggml-cpu.so; decider reuses semif's build (17.9 s).
• Every on.pull_request.paths entry was checked against the scope regex (22/22 as expected).
• Real PR file lists: chore: clean up remaining iii-stream and pubsub references (MOT-3619) #1353 skips; (MOT-4951) feat(ide): coder::find-relevant — judge-ranked code discovery (jevgrep port) #1279, feat: worker-defined chat mentions (@kanban, @session, @trace) #1334, refactor(judge-provider,judge-typesafe,judge-clef,judge-decider,judge-semif,judge-laya): MOT-5333 follow-ups (MOT-5342) #1356, feat(judge-openai,judge-provider,judge-typesafe): OpenAI Decisions judge provider (MOT-5333) #1346 run the engine.
• The grep -E here-string was run under -eo pipefail like the runner.
• This PR touches the workflow, so its own run takes the full engine path. That run gives the CI timing.

https://claude.ai/code/session_01WQhvPNhJJpri8PUFLGuQ5s

Summary by CodeRabbit

• Tests
  • Rust engine tests now run only when changes affect relevant engine components; unrelated changes skip the suite.
  • Test runs use a shared temporary build directory and a minimal Linux x86_64 engine configuration.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
workers	Ready	Preview	Oct 9, 2026 1:19pm UTC
workers-tech-spec	Ready	Preview	Oct 9, 2026 1:19pm UTC

[github-actions]

skill-check — worker

0 verified, 83 skipped (no docs/).

Layer	Result
structure	✓
vale	✓
ai	✓
render	✓

Four for four. Nicely done.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: 0f8018ec-950e-4d27-bce6-4a185a99564c

---

📥 Commits

Reviewing files that changed from the base of the PR and between fd06275 and da1405d.

---

📒 Files selected for processing (2)

• .github/workflows/judge-e2e.yml
• crates/llama-native/build.rs

---

Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 0 remain after this review.

---

---

📝 Walkthrough

Walkthrough

The workflow checks changed paths and runs the Rust engine suite only when they match engine-related patterns. Those tests use a shared temporary Cargo target directory and III_LLAMA_CPP_MINIMAL=1. The build script uses that variable to control CPU variants and Vulkan for dynamic Linux x86_64 builds.

Changes

Real-engine test workflow

Layer / File(s)	Summary
Minimal Linux build configuration crates/llama-native/build.rs	The build script reruns when III_LLAMA_CPP_MINIMAL changes. For dynamic Linux x86_64 builds, it sets CPU variants and Vulkan to OFF when the variable is exactly 1, and to ON otherwise.
Scoped real-engine test workflow .github/workflows/judge-e2e.yml	The workflow compares the PR merge commit with its first parent and checks changed paths against engine-related patterns. It conditionally installs Rust and the pinned engine, then runs real-engine tests with a shared temporary Cargo target directory and III_LLAMA_CPP_MINIMAL=1. The workflow removes the Cargo cache and llama.cpp build-tools provisioning steps.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Other

---

Merge Risk: ⚪ Minimal · up to da140

The workflow still selects real-engine tests for this change. No issue identified here prevents merging after normal checks.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly summarizes the main changes: skipping engine tests for JS-only changes and reducing engine build time.
Docstring Coverage		Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 1 files. (1 skipped: 1 …
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

---

---

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

• Commit to this branch
• Create a new PR

---

📝 Generate docstrings

• Commit to this branch
• Create a new PR

---

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

---

---

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

---

_{A rabbit checks the paths at dawn,
Then starts the engine suite when matched.
A smaller build wakes up to run,
With CPU variants and Vulkan detached.
I hop beside the passing tests!}

_{Comment @coderabbitai help to get the list of available commands.}