chore(models): use GPT-6.1 Sol defaults with GPT-6 Luna fallback

[PeterDaveHello]

User description

Summary

Use gpt-6.1-sol as the default primary model and gpt-6-luna as the fallback. Keep GPT-5.x models selectable; existing explicit model configurations remain supported.

Update the commented reasoning/weak-model examples, routing examples, and generated configuration reference together. The second routing rule retains the Sol primary. The per-directory settings test supplies its own root fallback list instead of depending on the shipped default. The generated routing-rules description now retains the collision-repair prerequisite alongside the examples; a regression test prevents that warning from being lost.

Remove the added Default model choice note to avoid maintaining model prices and version-specific cost-map advice in the usage guide. Model-selection evidence remains in this description.

Only config.model and config.fallback_models change as active runtime settings. Model handlers, registries, dependencies, token limits, and retry implementation remain unchanged.

Why these defaults

Independent evaluation at the same reasoning effort

The Artificial Analysis Intelligence Index v4.3.2 provides a capability reference beyond token prices. Every row uses medium, matching the default config.reasoning_effort; scores and benchmark-task costs come from the same medium-variant page. Values checked October 12, 2026.

Role / comparison	Model and evaluation source	Reasoning effort	Intelligence Index ↑	Cost per index task (USD) ↓
Previous primary	GPT-5.6 Sol	medium	39	$0.50
Intermediate Sol release, not the previous default	GPT-6 Sol	medium	40	$0.25
New primary	GPT-6.1 Sol	medium	48	$0.21
Previous fallback	GPT-5.6 Terra	medium	30	$0.18
Previous Luna generation, not the previous fallback	GPT-5.6 Luna	medium	25	$0.02
New fallback	GPT-6 Luna	medium	30	$0.02

• Primary: 39 → 48 (+9 index points) versus the previous default, with approximately 58% lower measured task cost. Compared with GPT-6 Sol, GPT-6.1 Sol scores 8 points higher at approximately 16% lower task cost.
• Fallback: Luna matches the previous Terra fallback's displayed aggregate score (30 → 30) at approximately 89% lower task cost. Compared with the previous Luna generation, its score increases 25 → 30 (+5 points). Both Luna generations display $0.02 per task after rounding; that does not establish identical exact costs.

The index combines 10 evaluations: AA-Briefcase v1.1, GDPval-AA v2.1, AutomationBench-AA, Terminal-Bench 4.0, SciCode, Humanity's Last Exam, GDP.pdf, CritPt, AA-Omniscience, and AA-LCR v1.1. These span work tasks, coding/terminal use, knowledge/reasoning, and long-context reasoning; see the methodology. At that snapshot, SciCode and CritPt were marked "Under review" by the evaluator.

This is a composite index, not a coding-only score or a measurement of PR-Agent review quality. Equal aggregate scores do not imply equal capabilities on individual tasks, and index-point gains are not percentage improvements in review quality. The cited pages do not expose usable constituent scores for these comparisons, so no individual coding-benchmark superiority is claimed.

Task cost is the evaluator's weighted average, accounting for input, cache-hit, cache-write, reasoning, and answer tokens. Percentage reductions above use displayed rounded costs; they are not predictions of cost per pull request. Results can change as evaluations or pricing are updated.

Vendor debugging evaluation

OpenAI's GPT-6.1 Sol system card reports an internal research-debugging mean rubric score of 75.52%, versus 68.32% for GPT-5.6 Sol (+7.20 percentage points). It reports comparable KernelGen 1P performance, not a universal improvement.

These are vendor results, not independent replication or the same medium-normalized comparison as the table above. They do not directly measure PR-Agent review quality or establish that Luna outperforms Terra. Evaluation rationale stays in the commit/PR description rather than requiring ongoing benchmark maintenance in the usage guide.

Published token prices

Standard API prices, USD per million tokens, checked October 12, 2026:

Model	Input	Cached input	Output
Previous primary: GPT-5.6 (Sol)	$4	$0.40	$20
New primary: GPT-6.1 Sol	$2	$0.10	$10
Previous fallback: GPT-5.6 Terra	$2	$0.20	$12
New fallback: GPT-6 Luna	$0.10	$0.01	$0.50

gpt-5.6 is the alias for GPT-5.6 Sol, so the migration comparison uses the actual previous defaults. Input/cached-input/output unit prices decrease by 50%/75%/50% for the primary and 95%/95%/approximately 95.83% for the fallback.

These are Standard rates for prompts with no more than 272K input tokens. GPT-5.6 Sol's quoted prices include its current promotional pricing, published as available at least through November 21, 2026. Token usage, reasoning effort, cache writes, long-context pricing, processing tiers, and regional premiums affect the final bill; lower unit prices do not guarantee lower total cost.

Compatibility and known limitations

• Routing remains disabled by default. The normal default chain is Sol → Luna. If the Luna routing example is enabled on the current implementation, it can retry Luna instead of restoring Sol. The separate collision repair is tracked in #4064; this PR does not include or claim that repair. The guide warns against enabling that example without the repair.
• Leave config.model_weak unset to use the normal chain for weak-model calls. Explicitly enabling the commented Luna weak-model example with Luna as fallback creates duplicate Luna attempts; the warnings document this limitation rather than changing retry behavior.
• Existing native OpenAI handling normalizes GPT-6 minimal to low and Sol's unsupported none to low; Luna supports none. The guide clarifies these settings. Usable context remains capped by config.max_model_tokens.

Validation

• Unit suite: 13,435 passed, 33 skipped.
• Ruff and file-scoped pre-commit checks passed.
• Generated configuration reference matches the generator output.
• Documentation build passed; all 32 referenced documentation URLs and static-asset links across 48 built pages resolve.

---

PR Type

Enhancement, Documentation, Tests

---

Description

• Default to GPT-6.1 Sol with Luna fallback.

• Align model examples and configuration reference.

• Document pricing limits, compatibility, and retry caveats.

• Decouple fallback preservation tests from shipped defaults.

---

Diagram Walkthrough

flowchart LR
  defaults["Shipped model configuration"]
  primary["GPT-6.1 Sol primary"]
  fallback["GPT-6 Luna fallback"]
  defaults -- "selects" --> primary
  primary -- "on failure" --> fallback

Loading

File Walkthrough

	Relevant files
Configuration changes	configuration.toml Adopt Sol primary and Luna fallback defaults                          pr_agent/settings/configuration.toml Set config.model to gpt-6.1-sol and config.fallback_models to ["gpt-6-luna"]. Update commented reasoning-model and weak-model examples; warn about duplicate Luna retries. Align disabled routing examples and note the required routed-fallback collision repair. +7/-5
Documentation	changing_a_model.md Document new defaults and model migration caveats                docs/docs/usage-guide/changing_a_model.md Document new defaults, comparative token pricing, and LiteLLM pricing metadata limitations. Explain retaining previous GPT-5.6 defaults, reasoning-effort normalization, and context limits. Warn about weak-model fallback duplication and the routing collision repair requirement. Update routing examples to select Luna for small PRs and retain Sol for the second rule. +43/-5    configuration_reference.md Align configuration reference with new model defaults        docs/docs/usage-guide/configuration_reference.md Synchronize documented primary and fallback defaults with configuration.toml. Update routing examples to use gpt-6-luna and gpt-6.1-sol. +3/-3
Tests	test_per_directory_settings.py Isolate root fallback preservation from shipped defaults  tests/unittest/test_per_directory_settings.py Give the root configuration fixture an explicit two-model fallback list. Assert nested settings preserve root fallbacks rather than relying on shipped defaults. Clarify that the assertion also detects replacement by default fallbacks. +5/-2

---

---

⚙️ Agent run details

• Model: openai/gpt-6.1-sol
• Tokens: 7,323 in / 615 out / 7,938 total
• Time cost: 14.4s
• AI calls: 1

[the-pr-agent-bot]

PR Code Suggestions ✨

No code suggestions found for the PR.

---

⚙️ Agent run details

• Model: openai/gpt-6.1-sol
• Tokens: 6,806 in / 62 out / 6,868 total
• Time cost: 6.9s
• AI calls: 1

[Copilot]

🟡 Changes recommended

The generated reference omits a required warning and presents a currently unsafe routing example.

1 open finding

• Preserve warning for unsafe Luna fallback routing example · New

What changed in this PR

Updates PR-Agent’s default model chain to GPT-6.1 Sol with GPT-6 Luna fallback.

Changes:

• Updates active model defaults and routing examples.
• Documents pricing, reasoning behavior, compatibility, and collision limitations.
• Makes per-directory fallback tests independent of shipped defaults.

File	Description
pr_agent/​settings/​configuration.toml	Sets new defaults and examples.
docs/​docs/​usage-guide/​configuration_reference.md	Updates generated defaults and routing example.
docs/​docs/​usage-guide/​changing_a_model.md	Documents migration and compatibility details.
tests/​unittest/​test_per_directory_settings.py	Adds explicit root fallback fixtures.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟢 Approved

The defaults, documentation, generated reference, and regression tests are consistent, with known routing limitations clearly documented.

0 open findings

1 resolved since last review

• Preserve warning for unsafe Luna fallback routing example

🧠 Review effort: Balanced

[naorpeled]

Can you please drop this change? 🙏
We'll soon upgrade litellm so it won't be necessary