feat: GPU acceleration, rule-based routing with context-window safety, Granite big-window fallback + gpt-oss tuning#124
Merged
Conversation
Controlled comparison of 8 local GGUF models x 2 prompt versions for the plugin's .ai.md code-summarization task (config+provider scope, 16K context, CPU / Ryzen 7 5800H), plus the full-project 30B baseline. 17 output trees (431 .ai.md) under docs/ai-index-benchmark/outputs with COMPARISON.md (scored matrix, findings, recommendation) and README.md (methodology + reproduce commands). Key findings: - No reasoning leakage or keyword hallucination in any cell (llama.cpp strips Qwen <think> and gpt-oss harmony server-side; output is clean). - Qwen3-Coder-30B-A3B validated as the default; Granite-4.0-H-Tiny is the standout new pick (~4x faster, Apache-2.0, accurate). - Prompt v2 cuts verbosity/accessor bloat but regresses with code fences on code-specialized models -> not yet shippable. - Reasoning/thinking is a pure throughput tax here (non-thinking models win). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
… corroboration) Expand docs/ai-index-benchmark/COMPARISON.md and link it from the root README: - §5 per-model pros & cons table (source-grounded) - §6 source-faithfulness deep-dive on AiGenerationConfig.java - §7 recommendations incl. "max precision, time no object" -> gpt-oss-20b and the best-for-large-files pick - §8 per-file model matrix: 6 archetype files x 8 models, scored vs source (gpt-oss-20b most faithful, 5/6; Qwen2.5-Coder-7B wins the trivial enum) - §9 external corroboration from a web research pass: structural findings (speed<->active params, reasoning-as-throughput-tax, hallucination/tooling categories) corroborated; accuracy ranking is novel/unverifiable; only 2/8 models have published IFEval Fixes: DeepSeek-Coder-V2-Lite license corrected to the use-restricted custom DeepSeek Model License (not "commercial-OK"). Root README "Recommended Models" now links the benchmark and lists the validated picks. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
Two CPU-throughput optimizations for the llama.cpp provider, plus the prompt restructuring they enable: - cache_prompt: the model is loaded once and the shared prompt-template prefix's KV is reused across files (pinned to one slot); only the per-file source is re-prefilled. New per-model `cachePrompt` flag (default true) threaded through AiGenerationConfig -> AiModelDefinition -> AiModelDefinitionSupport -> LlamaCppJniConfig -> AbstractAiIndexMojo -> provider. Benchmarked ~10% faster (1055s -> 950s, 12 files, Qwen3-Coder-30B @ 16K); reuse is logit-exact. - System/user prompt split: static instructions now go in the SYSTEM message and the variable file name + content in the USER message. AiPromptSupport gains systemPrompt(key) and userMessage(file, body); buildPrompt remains a length proxy for trimming. All prompt templates drop their %s placeholders and end with a sentence describing the user message. Verified end-to-end (12/12 files, zero %s/think/harmony/fence leakage). - Context default lowered to 16384 (64K KV caused paging on CPU); pinned slot API requires net.ladenthin:llama 5.0.3-SNAPSHOT. PIT: cachePrompt getters/setters + toConfig copy fully covered; AiPromptPreparation Support made package-testable, fully covered, and added to the 100% mutation gate (config 76/76, prompt 36/36). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
Add §10 to the benchmark comparison: prompt-cache A/B on Qwen3-Coder-30B @ 16K (12 files, file phase) shows ~10% speedup (1055s -> 950s) with unchanged output, and notes that this is the entire cacheable share (source prefill + decode are irreducible, so a system/user split adds no further speed). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
…ort=low) Set reasoning_format=AUTO on the model so reasoning channels (Qwen <think>, gpt-oss harmony analysis) are parsed into reasoning_content and message content stays clean (final channel only). Add reasoning_effort=low to the chat-template kwargs so gpt-oss spends minimal chain-of-thought on these short structured summaries (the gpt-oss default is medium); non-gpt-oss templates ignore it. Validated on a gpt-oss-20b real run (config+provider, 16K): 0/12 harmony-marker leaks and ~85 s/file vs ~121-129 s/file at the medium default (~30% faster), confirming the JNI binding forwards chat_template_kwargs.reasoning_effort. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
Replace the hard-coded reasoning_effort="low" with a per-model `reasoningEffort` config field (default "low") threaded through AiGenerationConfig -> AiModelDefinition -> AiModelDefinitionSupport -> LlamaCppJniConfig -> AbstractAiIndexMojo -> provider, mirroring the cachePrompt plumbing. The provider sends it as the reasoning_effort chat-template kwarg only when non-blank (empty omits it, letting the model template default apply). gpt-oss honors low/medium/ high; other models ignore it. Exposed for benchmarking via the ai.reasoningEffort property on the EXP-gpt-oss-20B definition (override with -Dai.reasoningEffort). PIT: config package 80/80 (100%) including the new getter/setter and the toConfig copy. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
Replace the broken EXP-gpt-oss-20B definition (16K context / 1536 output
budget, driven by the ai.reasoningEffort property) with three ready-to-use,
self-contained presets: gpt-oss-20B-{small,medium,high}-c48k.
gpt-oss reasons in-band: the analysis channel counts against maxOutputTokens,
so a higher reasoningEffort truncates the final summary unless the output
budget grows with it. The presets scale the budget to the effort
(small/low=4096, medium=6144, high=8192) and are sized to handle large source
files (~100 KB / >1000 lines):
- contextSize 49152 (48K) holds a ~100 KB file (~30K tokens) plus prompt
plus the output budget;
- charsPerToken 3 matches dense Java (the 4-char default is too optimistic
for code and risks real context overflow on big files).
Validated medium-c48k on a 96 KB / 2470-line class: maxInputChars=125800,
truncated=0, natural stop at n_decoded=1198 (<<6144), no retries, complete
summary, BUILD SUCCESS.
Remove the now-unused ai.reasoningEffort property. Production wiring stays on
Qwen3-Coder-30B for the self-index.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
Add COMPARISON.md §11: low/medium/high reasoning_effort comparison (timing, reasoning share, per-file truncation evidence; high aborted as strictly dominated) and a "Clean runs & large files" section documenting the two independent limits (contextSize vs maxOutputTokens), the medium-c48k validation on a 96 KB / 2470-line file, and the CPU prefill-dominated cost reality for ~100 KB files. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
Replace the effort-tiered gpt-oss presets with three size-tiered ones, all at reasoningEffort=low (the benchmark shows low is best for dense code summaries): gpt-oss-20B-c16k -> files up to ~40 KB (typical files; ~1-2 min/file) gpt-oss-20B-c48k -> files up to ~125 KB (large files; ~25 min @ 100 KB) gpt-oss-20B-c96k -> files up to ~260 KB (huge files; ~80 min @ 250 KB) The variable that actually matters in a real project is the largest file you must cover, which sets contextSize (and thus the untrimmed input). All three keep charsPerToken=3 (dense Java measured ~4.2 chars/token; 3 keeps the char-based pre-trim token-safe) and gpt-oss sampling (temp 1.0, top-p 1.0, top-k 0). Hard ceiling is the 128K window (~480-500 KB of code); CPU prefill + decode time make ~250 KB the practical per-file limit. Both c48k (96 KB file) and c96k (253 KB file) validated end-to-end: no trim, natural stop, complete summaries, BUILD SUCCESS. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
…n + generator Expand COMPARISON.md §11: - context-window ceiling (n_ctx_train=131072 ~= 480-500 KB of code; CPU time is the practical limit); - the three size-tiered presets (c16k/c48k/c96k) with coverage and timing; - end-to-end validations for 96 KB (~25 min) and 253 KB (~80 min) fixtures (truncated=0, natural stop, complete summaries); - cost reality (prefill O(n^2); decode ~1 t/s at 96K); - strategy for repos with mixed file sizes: routing is by extension not size, contextSize costs RAM not per-small-file speed, so use one preset sized to the largest file or split into scoped runs; size-based auto-routing is a non-feature. Add docs/ai-index-benchmark/tools/generate-fixture.sh: parameterized, valid synthetic large-class generator used to produce the benchmark fixtures. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
Back the "contextSize costs RAM, not per-small-file speed" claim with a measured A/B: the same 10 KB file under gpt-oss with everything identical except contextSize (16K/32K/48K/96K). Projected KV+compute RAM rises ~linearly (~+400 MiB per +16K) while prefill (~26.6 ms/tok), decode (~9.2 t/s), and wall time (~160 s) stay flat — a wider window is RAM-only for small files. Confirms the "one preset sized to the largest file" recommendation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
Add AiGenerationTimeEstimator: a rough, reference-CPU model of generation time. Generation is not a constant tokens/second — prefill per-token cost grows linearly with prompt length n (attention is O(n) per token), so total prefill is O(n^2), and decode slows as the context grows. Fitted to three end-to-end measurements (reproduced to within ~0.5 %, asserted by a regression test): prefill_ms(n) ~= 24.4*n + 0.000674*n^2 n = prompt tokens decode_ms(n, out) ~= out*(56.8 + 0.01568*n) n ~= source_chars/4.2 + ~400 (template) AiFieldGenerationSupport now logs a per-file line before generating, e.g. "Processing file '...' (96 KB source, ~24009 tokens) — estimated ~25 min (rough; ...)", so a large file's multi-minute block is no longer a surprise. The estimate is clearly marked rough/hardware-specific (a GPU or different CPU/model shifts the coefficients; only the quadratic shape is universal). Make gpt-oss-20B-c96k the default preset and document why: the measured A/B (§11) shows a wider window costs only KV RAM, not per-file time, so the widest practical window covers every file up to ~250 KB with no trim/truncation while small files stay just as fast. Downshift to c48k/c16k only to save RAM. AiGenerationTimeEstimator added to the PIT gate (100% mutation coverage). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
… in §11 Document the quadratic prefill timing model with its fitted coefficients and the three measured points it reproduces, explain why throughput "shrinks" as the prompt grows (per-token cost rises with n; decode slows over a longer context), mark c96k as the default preset with the RAM-vs-time justification, and note the per-file ETA the plugin now logs via AiGenerationTimeEstimator. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
…ming model Root README: add a feature bullet for the per-file duration-estimate log and a "gpt-oss-20b presets, large files, and timing" section (size-tiered c16k/c48k/ c96k table, c96k default with the RAM-vs-time rationale, 128K window ceiling, and the O(n^2) prefill timing formula), pointing to COMPARISON.md §11. Benchmark README: point the gpt-oss bottom-line entry at reasoningEffort=low, the three presets (default c96k), and §11 for context limits / 250KB validation / timing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
Use a plain ASCII hyphen instead of an em-dash in the per-file processing log line so it renders cleanly in any console/CI encoding, and show "<1 KB" for sub-kibibyte sources instead of the integer-truncated "0 KB". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
Switch the selftest executions (file/package/project) from Qwen3-Coder-30B to gpt-oss-20B-c96k: the per-file accuracy leader, run at reasoningEffort=low and a 96K window so it covers files up to ~250 KB untrimmed. Accuracy is the priority; the slower speed is the accepted cost. Qwen3-Coder-30B stays defined as the throughput-optimal alternative. Comments updated to reflect the new active default. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
…lternative) Reconcile the recommendations across COMPARISON.md §7, the benchmark README bottom line, and the root README: gpt-oss-20B-c96k is the production default (precision-first, large-file-ready via the 96K window), with Qwen3-Coder-30B reframed as the throughput-optimal alternative. Benchmark findings unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
Objective benchmark (24 KB ground-truth fixture, 2 reps): at an adequate output budget, low/medium/high all produce complete, equally accurate summaries (9/9 sections, count error 0). Higher effort buys no quality on this extraction task — it only decodes 4–8× more reasoning tokens (2–3× slower). Truncation is a decode-budget function of effort (low fits 2048; high needs ≥~5000). Confirms reasoningEffort=low as the default. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
Objective benchmark (24 KB ground-truth fixture, low effort, 2 reps): temp 0.0 and 0.3 hit the exact method count every time and temp 0.0 is fully deterministic (reproducible .ai.md — no incremental-diff churn); temp 0.7/1.0 introduced wrong counts (1, 40 vs 26) in some reps, at no speed benefit. For code extraction, temp 0.0/0.3 beats the card's 1.0. Logged as deferred decision D1 (candidate default change, pending more reps). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
…ale finding Four new timing points (25/50/100/150 KB) confirm prefill_ms(n) = 24.4n + 0.000674n^2 within +1.1%..+2.6% (consistent ~+2% over-prediction = safe ETA bias; constants kept). New finding: count accuracy degrades with scale even at temp 0 — exact to 128 methods (100 KB) but off by 23% at 195 methods (150 KB), so the earlier miscount is both a temperature and a scale effect. Hard-ceiling probe (~450 KB ≈ 3 h prefill) deferred; 250 KB stays the validated practical limit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
Linear fit over 7 real source files (each indexed in its own process to avoid cache_prompt undercounting): promptTok ≈ 713 + bytes/4.81, i.e. real Java is ~4.8 chars/token with ~713 template tokens — vs the synthetic fixture's 4.2 and the estimator's defaults (4.2 / 400). The estimator therefore over-estimates time by ~12% (safe direction). Logged as deferred decision D2 (constant refinement). Also documents the cache_prompt multi-file undercount methodology pitfall. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
…iscount + summary At 150 KB / 195 methods (temp 0) the base prompt rounded to a wrong "150"; a count-focused variant (explicit "give exact family count + range" + "COUNT EXACTLY" rule) reported the exact 195 — same speed, same completeness. Adds the TL;DR summary table and deferred decision D3 (fold the count instructions into the production prompt). D1 (temp 0) + D3 together fix the structural-count weakness common to all benchmarked models. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
The objective scorer used across all five phases: parses an .ai.md plus its llama.cpp run log into metrics (section completeness, ground-truth count error, prefill/decode timing). Companion to generate-fixture.sh; referenced from the tuning findings doc. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
Plumb a min_p sampling threshold end-to-end (AiGenerationConfig / AiModelDefinition / AiModelDefinitionSupport / LlamaCppJniConfig / AbstractAiIndexMojo / provider .withMinP), mirroring the existing reasoningEffort/cachePrompt pattern. Default 0.0 = disabled (neutral for existing models). This is the prerequisite for the gpt-oss sampling change (D1): without it, lowering temperature would inherit the llama-server min_p default (0.1) instead of an intended value. PIT-gated config classes kept at 100% with added round-trip/propagation/default tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
…t prompt After a two-round web fact-check (docs/ai-index-benchmark/gpt-oss-tuning.md): D1 (sampling): the three gpt-oss presets move from temp 1.0 to temperature 0.7 + min_p 0.05 (top_p 1.0, top_k 0 kept). NOT temp 0 — greedy/temp 0 triggers "reasoning blackhole" repetition loops in ~81% of probed gpt-oss prompts; temp 0.7 with min_p 0.05 as the primary, confidence-scaled truncation is the faithful-but-safe middle ground vs the card's 1.0. Preset comment updated. D3 (prompt): file-body-java gains a Public-API rule to enumerate a near-identical member family in the analysis channel and emit only the exact total + range in the final answer, plus a global "COUNT EXACTLY" rule. Fixes the large-file miscount (Phase 3: wrong "150" -> exact "195" at no cost). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
… +15% margin) D2: refit the duration estimator on real source. ESTIMATION_CHARS_PER_TOKEN 4.2 -> 4.8 and PROMPT_TEMPLATE_TOKEN_OVERHEAD 400 -> 700 (linear regression over 7 real repo files, Phase 5), plus a +15% display margin (ETA_SAFETY_MARGIN) since 4.8 removes the accidental conservatism 4.2 provided. The quadratic prefill model is unchanged (the web review confirmed it is already the correct linear+quadratic SWA-aware shape). Tests updated; PIT 100%. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
Record the outcome of the external fact-check and that D1/D2/D3 are implemented: D1 revised (temp 0.7 + min_p 0.05, NOT temp 0 — greedy is unsafe for gpt-oss), D2 (4.8/700 + 15% margin), D3 (count-exact prompt). Document deferred items (reasoning/think budget, conditional effort, --swa-full/build-pin) and the must-re-test list. Reconcile COMPARISON.md §11 (real-Java 4.8/700 timing constants; chars/token note). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
…chine re-test Third external web review challenged the OPTIMALITY of the shipped D1/D2/D3 config (verdict: sound and safe, not provably optimal; corrected the "quant fever" = quantization misconception). On-machine re-test of the shipped config (16 real files ×2 reps + 5 ground-truth fixtures): - T1 blackhole/safety: PASS — all complete, 0 truncations, 0 loops (temp 0.7 + min_p 0.05); only the 0.5 KB enum showed 6-10% duplicate lines (harmless). - T2 count faithfulness: PASS — exact at 26/128/195 methods, low AND medium effort; the previously-"150" 195 case is robustly fixed by D3. - T3 ETA accuracy: PARTIAL/conservative — within ±15% 4/8, under-promised only 1/8 (over-predicts small files; safe direction; matches the "rough" label). Records the read-only DRY check (binding has DRY only at model level; per-request withDry specified for a separate agent) and the optional enhancements E1-E6. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
…ault off Plumb a top-n-sigma sampling threshold end-to-end (AiGenerationConfig / AiModelDefinition / AiModelDefinitionSupport / LlamaCppJniConfig / AbstractAiIndexMojo / provider .withTopNSigma), mirroring the min_p pattern. Default -1.0 = disabled (neutral for existing models). Enables the E6 A/B (top-n-sigma is temperature-invariant, a candidate alternative to min_p for reproducible extraction). PIT-gated config classes kept at 100%. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
Measured A/B (temp 0.7, low effort, fixtures 26/128/195): top-n-sigma 1.0 ties min-p 0.05 on small/medium files (count error 0, complete) but FAILED the 150 KB / 195-method case — emitted only the lead, 0/9 sections (early stop, no loop, no trim), where min-p produced the full exact summary. Decision: keep min_p 0.05 default; do not adopt top-n-sigma (no accuracy gain, riskier on large files). The top_n_sigma plumbing stays in, default off. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
…, priority, skip, explicit fallback Replace the single-provider-per-run model with a plan-then-execute flow so one `generate` run can route each file to its own model AND prompt, shown up front as a tree grouped by model. Routing rules (<fieldGeneration>): filters fileExtensions + min/maxFileSizeBytes + min/maxLines, a priority (highest wins; ties by declaration order), and a kind: route (prompt+model), explicit <fallback>, or <skip> (ignore matching files; a high-priority skip overrides routes and the fallback). Selection picks the highest-priority matching non-fallback rule; else the explicit fallback; else the file is unmatched. Validation (fail-fast): at most one fallback; a non-fallback rule needs >=1 filter; route/fallback need prompt+model; fallback can't be skip. Flow (GenerateMojo): validate rules -> collect candidates -> classify into an AiIndexPlan (routes grouped by aiDefinitionKey, skips, unmatched) -> log the tree -> unmatched aborts the build -> planOnly (aiIndex.planOnly) stops here -> else load each model once (sequentially, one resident at a time -> bounded RAM), index its files, close. SourceFileIndexer split into collectCandidates/classify/indexFile; AbstractAiIndexMojo.buildLlamaCppJniConfig(key) builds a provider per model id (fixes the latent bug where only fieldGenerations.get(0)'s model was ever loaded). Different aiDefinitionKeys can be the same gguf with different params (e.g. ctx), each its own group/load. pom: the file-body-fallback rule now marked <fallback>true</fallback> (implicit "no-filter = fallback" is gone). 240 tests; PIT 339/339 (100%). planOnly self-index verified. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
Describe the plan-then-execute generate flow, the <fieldGeneration> routing rules (extension/size/lines filters + priority + route/skip/explicit-fallback), the per-model-group execution (one model load per aiDefinitionKey), the planOnly preview, and the new AiFieldGenerationSelector/AiIndexPlan components. Add planOnly to the GenerateMojo parameter table. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
… total The plan tree is now a Markdown document (one log line, copy-pasteable): a summary with the grand-total estimate, then per model a heading + a | File | Prompt | Est. | table and a per-model time subtotal, then skipped/unmatched lists. Each routed file carries a rough estimated generation time (AiGenerationTimeEstimator over the file size; not precise by design); times are summed per model and overall. AiIndexPlan.Entry gains estimatedSeconds; addRoute takes it; renderTree -> renderMarkdown; SourceFileIndexer.classify computes the estimate per file. ASCII markers for CI-log safety. 243 tests; PIT 339/339 (100%). planOnly verified: "Total: 12 files, est. ~31 min" + per-file rows. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
Add an optional <id> to each <fieldGeneration> routing rule (e.g. java, sql, fallback, skip-generated). The plan markdown table now has a Rule column (| File | Rule | Prompt | Est. |) so you can see WHICH rule routed each file, not just the prompt — useful when several rules share a prompt but differ by filter/model/priority. Unset id renders as "-". The id also surfaces in AiFieldGenerationSelector.validate error messages (config @tostring). The pom's generate rules are labelled java/sql/fallback as an example. 244 tests; PIT 340/340 (100%). planOnly verified: Rule column shows "java". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
…leaves) Replace the flat per-rule filters (fileExtensions/min-max size/min-max lines) with a composable <condition> tree, evaluated against a file's facts. Not backward compatible by design. Each <condition> node sets exactly one of: combinators <and>/<or>/<not>, or leaves <extensions>, <size> (AiRangeCondition min/max, bytes), <lines> (AiRangeCondition), <modifiedAfter>/<modifiedBefore> (ISO-8601 instant vs file mtime), <pathGlob> (base-relative glob, reusing AiSourceExcludeFilter). Nesting is native Maven binding (named fields, no implementation= attributes); new leaf kinds = one field + one evaluator branch. New gated config classes: AiCondition (recursive POJO), AiRangeCondition, AiFileContext (the evaluated facts), AiConditionEvaluator (matches/validate/ usesLines). AiFieldGenerationConfig now carries a <condition>; the selector takes an AiFileContext and validates that non-fallback rules have a (valid) condition, the fallback has none, etc. SourceFileIndexer.classify builds the context (line count read lazily only when a lines leaf is present). pom example rules migrated to <condition>. ProjectIndexer.copyFieldGeneration drops the removed field. 273 tests; PIT 421/421 (100%, new classes added to the gate). planOnly verified: .java routed via condition, 0 unmatched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
Update the Phase-1 routing description and the rule-based-routing principle to the composable <condition> tree (extensions/size/lines/modifiedAfter/modifiedBefore/ pathGlob with and/or/not), replacing the old flat per-rule filters, and add the AiCondition/AiConditionEvaluator component row. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
AiFieldGenerationConfig.getId() is now @nullable; assign it to a local before the null check so the Checker Framework can refine it (two getId() calls would not be refined). Required for the -Werror build. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
…><condition>
Maven derives a collection's item element name from the field name's singular, so a
List<AiCondition> field named and/or required the items to be <and>/<or> — verified
that <and><condition>…</condition></and> fails to bind ("Cannot find 'condition'
in AiCondition"). Wrap the combinator children in AiConditionGroup (field
`conditions` -> items <condition>), so the syntax is:
<and><conditions><condition>…</condition><condition>…</condition></conditions></and>
(<not> still nests a single condition directly). Evaluator/validate/usesLines read
group.getConditions() with null/empty guards. New gated class AiConditionGroup
(added to PIT targetClasses). Runtime-verified via planOnly that nested
and/conditions + size + not/pathGlob bind and route (17 files, 0 unmatched).
README routing example + CLAUDE.md updated to the group syntax; .gitignore ignores
the benchmark scratch dirs (siteGusto/siteMistral3/siteQwenBRUTAL). 277 tests;
PIT 426/426 (100%).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
…t-oss work Update the stale PIT note (was 24 classes/200 mutations) to the current 426/100% incl. the new condition config classes, and add a Done entry for the rule-based routing + general-purpose condition tree, per-file time log, and quant menu. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
…ld default) Per-model gpuLayers on the aiDefinition, plumbed through AiGenerationConfig / AiModelDefinition / AiModelDefinitionSupport / LlamaCppJniConfig (now 24 fields) / both buildLlamaCppJniConfig sites, applied in the provider's model() only when >= 0 (modelParameters.setGpuLayers). -1 (default) leaves the binding/native-build default so a CPU build stays CPU and a GPU build keeps its own default; 0 forces CPU even on a GPU build; >0 offloads that many layers — the control needed for partial offload when a model does not fit in VRAM (e.g. gpt-oss-20b on 8 GB). Only effective with a GPU native (cuda13/vulkan classifier). 280 tests; PIT 427/427 (100%). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
…assifier
Make the net.ladenthin:llama native build selectable via a property
(llama.classifier, empty = the CPU/Ninja main jar) and add two profiles:
-P gpu-cuda -> cuda13-windows-x86-64 (NVIDIA, ~4.5x CPU decode; needs the CUDA
toolkit bin\x64 on PATH for cudart64_13/cublas64_13)
-P gpu-vulkan -> vulkan-windows-x86-64 (AMD + NVIDIA; one-time shader warmup)
The gpt-oss c16k/c48k/c96k presets gain <gpuLayers>${ai.gpuLayers}</gpuLayers>
(default -1 = leave build default; set -Dai.gpuLayers=N for partial offload on
limited VRAM). Verified: empty classifier resolves the main jar; -P gpu-cuda loads
the CUDA device in the native test (decode ~427 t/s). README documents the GPU
options + the CUDA PATH requirement. OpenCL deliberately omitted (no NVIDIA support
in llama.cpp).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
Verified that the gpu-cuda/gpu-vulkan profiles only affect this repo's reactor build (no POM flatten -> the llama.classifier property resolves to the CPU default downstream), so they do NOT change the native used when the published plugin runs in another build. The reliable GPU mechanism for actual indexing is the runtime override net.ladenthin.llama.lib.path (loaded before the bundled native), verified to load CUDA (~417 t/s) with the default CPU jar on the classpath. README now documents lib.path as the primary GPU path + the profiles as reactor test/bench. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
…endency GPU path Adds two opt-in knobs alongside gpuLayers, plumbed through the full chain (AiGenerationConfig -> AiModelDefinition(+Support) -> LlamaCppJniConfig -> provider) and applied as the binding's --main-gpu / --device: - mainGpu (default -1 = leave default): pick the primary GPU on multi-GPU hosts. A CUDA build only enumerates NVIDIA devices (single-NVIDIA host needs nothing); a Vulkan build enumerates every GPU (an integrated GPU is often device 0), so the default may select the slower one -- mainGpu=1 forces the discrete GPU. - devices (default "" = leave default): explicit comma-separated backend device names (e.g. Vulkan1); takes precedence over mainGpu. Both default to neutral (no setter call), so behaviour is unchanged unless set. Exposed on the gpt-oss presets via ai.mainGpu / ai.devices and as <mainGpu> / <devices> on any model definition. README: document the clean GPU enablement path -- add the GPU classifier as a <dependency> of the plugin so the loader extracts it from the plugin classpath (no lib.path needed); lib.path remains the no-POM-change alternative. Verified empirically that the loader extracts the GPU native from the classpath jar without lib.path (CUDA, ~431 t/s). 286 tests pass (1 native skipped); PIT 107/107 (100%) on the changed config classes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
…U-ready
- New ai.model property (default gpt-oss-20B-mxfp4); the generate/aggregate
executions now reference ${ai.model} instead of the hardcoded c96k key, so the
default model is one line and overridable with -Dai.model=<key>. mxfp4 is the
native format: same ~12 GB size and decode speed as Q4/Q5/Q8 (experts are MXFP4
regardless) but no expert double-quantization -> best quality. Verified via
planOnly: plan routes all 17 files to gpt-oss-20B-mxfp4, 0 unmatched.
- Wire ${ai.gpuLayers}/${ai.mainGpu}/${ai.devices} into all five quant-menu presets
(mxfp4/Q4_K_M/Q5_K_M/Q8_0/F16), so any selectable model is GPU-ready.
- Document that ai.gpuLayers=-1 means AUTO-FIT (llama.cpp fits as many layers as the
card's free VRAM holds) -- the robust "runs on any card" default, not a fixed
pessimistic count. Measured on RTX 3070 8 GB: auto-fit ~29 decode t/s vs ~8 CPU.
- Record the GPU cross-check (quant within noise; CUDA > Vulkan-discrete >
Vulkan-iGPU ~ CPU; OpenCL drops NVIDIA) in the quant-menu comment + README.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
…plan code The full `mvn verify` SpotBugs gate (Max+Low + fb-contrib 7.7.4) surfaced 44 findings in the rule-routing / condition-tree / plan classes added earlier this session (AiCondition(+Group), AiFieldGenerationConfig, AiConditionEvaluator, AiFieldGenerationSelector, AiIndexPlan(+Entry), AiFileContext, AiRangeCondition, AiGenerationTimeEstimator) - that code had passed tests + PIT but never the full verify. All 44 are fb-contrib bytecode-shape opinions on build-time, single- threaded, plugin-internal beans/DTOs, the same families already suppressed with rationale elsewhere in this filter: - EI_EXPOSE_REP/REP2 on the Maven @parameter config beans + the throwaway AiIndexPlan DTO (the "representation" is the configured tree / internal plan, never shared with untrusted callers; Entry.rule intentionally aliases the rule) - IMC_IMMATURE_CLASS_NO_EQUALS / NO_TOSTRING / WRONG_FIELD_ORDER - CE_CLASS_ENVY, OCP_OVERLY_CONCRETE_PARAMETER, OPM_OVERLY_PERMISSIVE_METHOD, UCPM_USE_CHARACTER_PARAMETERIZED_METHOD, WEM_WEAK_EXCEPTION_MESSAGING, RCN_REDUNDANT_NULLCHECK_OF_NONNULL_VALUE Each block is scoped to the specific classes + patterns with a written rationale. No production behaviour change. Full `mvn verify` now BUILD SUCCESS (SpotBugs 0). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
…budget still unproven Re-ran both E2 arms on build 11 via CUDA (mxfp4, ~27 KB fixture, temp 0.7, high effort, maxOutputTokens=2048, ctx 16384): - Arm A (budget -1): non-empty body (2615 chars, ~12 headings), decode stopped at 1604 tokens BELOW the 2048 cap -> the original "reasoning blackhole / 0-of-9 empty body pinned at the cap" no longer reproduces on build 11. - Arm B (budget 1024): non-empty (2055 chars), 1350 decode tokens; no clean truncation at the budget -> reasoning-budget still not a demonstrable safety rail. Conclusion unchanged: reasoningBudgetTokens stays opt-in/default -1, low effort stays the shipped default. Only material update: high effort is no longer outright broken for larger files on build 11. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
…n, gitignore)
Repo-wide consistency audit fixes:
- Default model: README, docs/ai-index-benchmark/README.md + COMPARISON.md, and the
pom DEFAULT comment said gpt-oss-20B-c96k; the actual default is now ai.model=
gpt-oss-20B-mxfp4 (native-MXFP4 swap of the benchmarked c96k/UD-Q4_K_XL, same 96K
window, quant within noise per E5). All updated + note the -Dai.model override.
- Version: CLAUDE.md said 1.0.0; aligned to pom 1.0.1-SNAPSHOT.
- pom charsPerToken comment cited the stale ~4.2 chars/token (synthetic fixture);
real Java is ~4.8 (D2), which the ETA estimator already uses. Comment corrected
and disambiguated from ESTIMATION_CHARS_PER_TOKEN.
- .gitignore: ignore .idea/ (was untracked) and guard generated AI-index output
under src/site/ai/ while keeping the tracked empty.md placeholder.
Audit also confirmed clean: mainGpu/devices wired symmetrically (26-field
LlamaCppJniConfig), all 8 gpt-oss presets carry the GPU knobs, ${ai.model} used in
every execution, 95/95 license headers, no TODO/retry remnants, GPU profiles + PIT
config consistent. No code/test change in this commit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
The routing plan now shows, per file, whether it fits the routed model's context window, and by default fails the build when a file would be trimmed - so oversized files are routed to a larger-context model instead of silently losing content. - AiInputWindowCalculator (new, pure, PIT 100%): single source of truth for the trim threshold (max input chars + availableSourceChars + exceedsWindow). Mirrors the runtime formula exactly. AiFieldGenerationSupport now delegates its maxInputChars math to it, so plan prediction == run behaviour. - SourceFileIndexer.classify gains an overload that, given the model + prompt supports, computes each routed file's window fit (source chars vs budget) and records it in the plan. The 2-arg overload (no window check) is unchanged. - AiIndexPlan: per-entry window fit, a "Window" column (ok / "(!) X>Y"), an "Over window" section listing offenders + their model/budget, a summary count, and windowExceededCount(). - GenerateMojo: new aiIndex.failOnWindowExceeded (default TRUE) - after the plan is printed, the build fails (planOnly and real run alike) if any file exceeds its window; set false to trim with the existing warnOnTrim warning instead. - pom: example big-window fallback (phi-4-mini-bigwindow, 128K) + a high-priority 'big-window' size rule routing >275 KB .java files to it, so the default fail-on-exceed stays satisfiable for rare huge files. (Download/verify the Phi-4 GGUF before relying on it; the path is an example.) - Docs (CLAUDE.md, README): document the window check, the new parameter, and the fallback pattern. Verified: planOnly on this repo shows "0 over window" (all 17 files fit the 96K mxfp4 window); full mvn verify green (297 tests, SpotBugs 0); PIT 17/17 on the calculator. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
… no source magic) Per maintainer direction: when a file is larger than its routed model's window the build must ALWAYS hard-fail; the resolution is user configuration, never an automatic model choice in the source. Removed the aiIndex.failOnWindowExceeded toggle (and its trim-instead escape) so the check is unconditional. The only way to handle oversized files is a user-configured <fieldGeneration> rule with a size <condition> routing them to a larger-window model (the phi-4-mini-bigwindow + the big-window rule remain as a config example). Plan output, the mojo error message, and the docs (CLAUDE.md, README, POM comments) all now state this clearly. mvn verify green (297 tests, SpotBugs 0). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
After each file the run now logs an aggregate progress line, advancing by the running sum of each finished file's PLAN estimate over the grand total (the simple estimate, no re-estimation), e.g.: [######## ] 41% 7/17 files - est. ~17 min/~42 min done, ~25 min left (estimate) | ~20 min elapsed (actual) - AiProgressBar (new, pure, support layer, PIT 13/13): renders [##### ] 42% from a completed/total ratio; clamps to [0,1] and treats total<=0 as full. - GenerateMojo: tracks done/total estimated seconds + file counts + run start, and logs the bar with estimated time left and actual wall-clock elapsed after every indexFile (so estimate vs reality is visible live). - spotbugs-exclude: scoped OPM suppression for AiProgressBar.render (public API), matching the AiGenerationTimeEstimator rationale. - Docs: AiProgressBar in CLAUDE.md Key Components. mvn verify green (306 tests, SpotBugs 0); PIT 13/13 on AiProgressBar. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
formatDuration already prefixes "~", so the literal "~" before the remaining-time value produced "~~40 min left". Removed it: now "... done, ~40 min left (estimate)". Found via a live mock generate run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
… verified Replaces the phi-4-mini-bigwindow example with IBM Granite 4.0-H-Tiny as the big-window fallback for oversized files, chosen from a licensed/origin-filtered model search: - Apache-2.0 (same license family as gpt-oss), IBM (not China). - Hybrid Mamba-2/Transformer (9:1, NoPE): KV cache grows only LINEARLY with context, so a huge file fits ungekürzt even on an 8 GB GPU (a classic transformer's KV would blow up at 128K). 128K context (trained to 512K). Verified on this machine (RTX 3070, 8 GB, CUDA, build 11): the binding loads the Mamba-hybrid architecture; it ingested a generated ~482 KB / ~123,865-token Java file at the full 131072 context with NO OOM (prefill ~2560 t/s, decode ~97 t/s) and produced a correct Markdown overview. Measured ~4 chars/token on dense Java (-> charsPerToken 4). reasoningEffort left empty (Granite has no gpt-oss harmony channels). The big-window rule now routes >275 KB .java files to it; Granite's ~128K window covers up to ~500 KB. mvn verify green (306 tests, SpotBugs 0); planOnly shows 0 over window for this repo. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
…erified) Raises the granite-4.0-h-tiny-bigwindow context to 384K (393216) so the fallback covers source files up to ~1 MB, and sets charsPerToken to 3 (conservative: real Java is ~3.8 chars/token, so the window check over-counts tokens and never lets a file overflow the window at run time; the cpt-3 check admits up to ~1.16 MB). Verified on this machine (RTX 3070, 8 GB, CUDA, build 11): Granite's hybrid Mamba/Transformer KV grows only linearly, so it allocates a 384K window (even 512K) with NO OOM and summarized a ~995 KB / ~268,193-token Java file end-to-end (prefill ~1150 t/s, ~4 min one-time; decode ~45 t/s). Quality is best within Granite's validated 128K (~500 KB) and degrades gradually beyond it, but a whole-file summary beats a trimmed one. Files >~1 MB still hard-fail (split or add a larger preset). mvn verify green (306 tests, SpotBugs 0); planOnly shows 0 over window for this repo. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
Removed the large "Done (kept for history)" section (routing/condition tree, three-level index, layered restructure, all the static-analysis gates — recorded in git history + crossrepostatus). Pruned items that are now fully done (SpotBugs Max+Low gate, ArchUnit layered rules). Refreshed the PIT item (adds AiInputWindowCalculator + AiProgressBar; notes AiIndexPlan/AiConditionGroup still ungated) and added the current open items: pin llama off SNAPSHOT (release blocker), README polish (1.0.0 example + minP), and the >1 MB big-window note. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
Replaces the abbreviated "temperature, top-p / top-k, repeat penalty, threads" sentence (which omitted min-p and many other knobs) with a complete table of every <aiDefinition> field and its shipped default, grouped sizing / sampling / DRY / chat-reasoning / KV-cache / GPU. Defaults taken from AiGenerationConfig.DEFAULT_*. Notes that maxInputChars is derived (not a field) and that charsPerToken should stay at or below the model's real ratio. TODO.md: minP doc gap resolved. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
… snapshot repo) 5.0.3 is now published on Maven Central (main jar + sources/javadoc + all GPU classifier jars: cuda13-windows/linux, vulkan-windows, opencl-windows/android, msvc-windows, with .asc signatures). Pinned <llama.version> 5.0.3-SNAPSHOT -> 5.0.3 and removed the now-unneeded central-snapshots <repositories> block. Verified the build resolves 5.0.3 from Central and full mvn verify is green (306 tests, SpotBugs 0). Docs synced to 5.0.3 (CLAUDE.md dependency table, README install example); TODO.md drops the "pin off SNAPSHOT" release blocker (done). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLccGn2dcCk7vqvBCTFWHd
bernardladenthin
had a problem deploying
to
startgate
June 29, 2026 19:42 — with 
GitHub Actions
Error
GitHub Actions
Error
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
generategoal now plans before it runs: a composable<condition>tree (and/or/notoverextensions/size/lines/modifiedAfter/modifiedBefore/pathGlob) withpriority,skip, and one explicitfallbackroutes each file to a(model, prompt); a copy-pasteable Markdown plan (model → file → rule → prompt → window-fit → ETA) is logged, and each file's fit against its model's context window is checked up front. A file too large for its model hard-fails the build — the fix is configuration (route it to a larger-window model), never an automatic model choice. A per-file progress bar + ETA is logged during the run.gpuLayers(--gpu-layers,-1= auto-fit to free VRAM),mainGpu,devices;gpu-cuda/gpu-vulkanprofiles + parameterized native classifier; documented plugin-dependency andlib.pathenablement. Verified CUDA and Vulkan on an RTX 3070; OpenCL deliberately not offered (it drops NVIDIA).gpt-oss-20B-mxfp4(ai.model); opt-inreasoningEffort/reasoningBudgetTokens/minP/topNSigma/DRY/swaFull/cacheReuseknobs (all neutral by default); ETA estimator calibrated to real Java; quant menu + benchmark docs; empty-body retry mechanism removed.net.ladenthin:llamato the released 5.0.3 (dropped the SNAPSHOT + snapshot repo); SpotBugsMax+Lowgate cleared with scoped rationale; new pure classes at 100% PIT; docs synced (full<aiDefinition>parameter table, GPU section, TODO).Test plan
[Unreleased]to be filled as part of the 1.0.1 release stepLocal: full
mvn verifygreen — 306 tests, 1 skipped (JNI native), SpotBugs 0 bugs. Real-model behaviour verified on an RTX 3070 (CUDA/Vulkan; gpt-oss-20b + Granite big-window up to ~1 MB).Related issues / PRs
Checklist
CONTRIBUTING.mdandCODE_OF_CONDUCT.mdSECURITY.md)