You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
refactor(cli): ship one Eval runtime in the npm CLI package #3933
copyEvalMirror() in scripts/release-cli-package.mjs:482 copies the whole staged node_modules/@maka/eval tree to packages/eval inside the published CLI package, so every install carries the Eval runtime twice.
The mirror is not pure redundancy: experiment configurations reference container paths under it, for example /opt/maka-agent/packages/eval/dist/harbor-external-subject.js in packages/eval/experiments/terminal-bench-2.1-deepseek-v4-flash-eight-arm.json. Those entries are what keeps the second copy alive.
Desired outcome
Ship one Eval runtime. Keep node_modules/@maka/eval as the runtime and reduce the mirror to only the container entry points the experiment configurations actually resolve, instead of the entire tree.
Keep: standalone packaging, Eval behavior, and the container paths those experiments depend on. maka eval remains the only public Eval CLI.
The PR should list the exact entry points it keeps and show that no experiment configuration resolves a mirrored path outside that list.
Alternatives or workarounds
Keeping the full mirror keeps install size and, worse, two runtimes that can diverge between releases.
Proposed in #3618. The rule for this issue is one end-to-end deletion, one PR, and strictly fewer maintained concepts; moving the same idea behind a new facade or compatibility bridge does not count.
Claude verified the consumer claims below against current main. A human contributor must confirm them and own the resulting change.
Activity
liuxiaocs7 commentedon Aug 26, 2026
I'd like to work on this issue. Could you please assign it to me? Thanks!