feat: support value references in LLM provider and model fields

[yohamta0]

What

The root llm: block (and its step-level counterpart) only resolved ${...} references in system and base_url. provider and model were rejected at load time by the provider whitelist, so a workflow could not select its model from a parameter or environment variable.

params:
  - name: PROVIDER
    default: anthropic
  - name: MODEL
    default: claude-sonnet-4-6

llm:
  provider: ${params.PROVIDER}
  model: ${params.MODEL}
  system: You are helpful

steps:
  - id: ask
    action: chat.completion
    with:
      prompt: hello

Changes

• llm.provider, llm.model, llm.api_key_name and the array form's model[].provider, model[].name, model[].api_key_name are now value-resolved fields. system, base_url and messages[].content behave as before.
• A provider carrying a reference skips the load-time whitelist check; it is validated against the supported provider list at step start, after resolution. A bad resolved value fails the step with invalid provider: <resolved value>.
• llm.tools[] stays literal — a reference cannot select a tool DAG.
• The DAG-level provider whitelist was a hand-maintained copy of llm.ParseProviderType that had never picked up zai or opencode, so those were accepted on a step but rejected at the root. Both levels now share one check, and the JSON schema enum is realigned with it.
• Spec 003's value-resolution matrix is updated to match, along with its conformance tests.

Numeric fields (temperature, max_tokens, top_p) are typed as numbers in YAML and are not covered.

Test plan

• go test ./internal/core/... ./internal/runtime/...
• make lint
• End-to-end: dagu validate accepts the DAG above; running it with PROVIDER=notaprovider fails the step with invalid provider: notaprovider, confirming the reference is substituted before the provider is constructed.

Docs live in a separate repo and need a follow-up there.

---

Summary by cubic

Enable choosing LLM provider and model via ${...} references at runtime, with provider validation deferred until resolution. Steps now fail early if provider or model resolve to empty to avoid downstream API errors.

• New Features

  • Runtime resolution for llm.provider, string llm.model, and array-form model[].provider and model[].name.
  • Providers with ${...} skip load-time checks and are validated after resolution when the step runs.
  • JSON schema accepts references and adds aliases (openai-codex, zai, opencode, zhipu, zhipuai, glm).

• Bug Fixes

  • Reject provider or model that resolve to empty; inherited configs with no model now error at step start.
  • Unified provider validation; removed the stale DAG-level whitelist.
  • Kept llm.api_key_name literal per its env-var contract; llm.tools[] remains literal.
  • Updated Spec 003 to cover provider/model resolution.

^{Written for commit 229ecc2. Summary will update on new commits.}

Summary by CodeRabbit

• New Features

  • Added runtime value resolution for LLM providers and model names using parameter and environment references.
  • Expanded LLM configuration fields that support value resolution, including provider, model, endpoint, and API key name.
  • Added support for runtime-resolved providers in model configurations.

• Bug Fixes

  • Improved validation to accept deferred provider references while continuing to reject unsupported concrete providers.

• Documentation

  • Updated value-resolution guidance for LLM configuration fields.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[coderabbitai]

📝 Walkthrough

Walkthrough

LLM provider validation now accepts runtime references, additional LLM fields participate in value resolution, and chat execution resolves model providers and names before creating providers and running models.

Changes

LLM provider resolution

Layer / File(s)	Summary
Provider validation contract internal/cmn/schema/dag.schema.json, internal/core/spec/*.go, internal/core/spec/llm_test.go	Provider schemas accept expanded provider names and ${...} references; core validation defers referenced providers and validates concrete values consistently.
LLM field resolution surfaces internal/core/value_fields*, internal/core/spec/spec003_conformance_test.go, specs/003-value-resolution.md	LLM-level and model-level provider, model, name, endpoint, and API key fields are included in resolution coverage; tool lists remain literal.
Chat execution resolution internal/runtime/builtin/chat/executor*.go	The chat executor resolves model providers and names before execution, defers parsing referenced providers, and validates providers during creation.
Estimated code review effort: 3 (Moderate)	~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ChatExecutor
  participant RuntimeContext
  participant ProviderFactory
  ChatExecutor->>RuntimeContext: resolve model provider and name references
  RuntimeContext-->>ChatExecutor: resolved model entries
  ChatExecutor->>ProviderFactory: create provider for resolved provider
  ProviderFactory-->>ChatExecutor: provider or parse error
  ChatExecutor->>ProviderFactory: run model with resolved configuration

Loading

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly summarizes the main change: adding value references to LLM provider and model fields.
Description check	✅ Passed	The description covers the change, impact, and test plan, though it doesn't follow the exact template headings or include related issues/checklist.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/llm-config-variables

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@internal/runtime/builtin/chat/executor.go`:
- Around line 366-385: Update createProviderForModel to resolve each model’s
api_key_name via runtime.ResolveString before using it for credential lookup, so
parameter-based names resolve to the corresponding environment variable value
rather than being passed literally. Add a regression test covering a parameter
that resolves to an environment-variable name and assert the provider receives
that environment value.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 265f3200-b6e4-404b-b9b8-dbdb643cf891

📥 Commits

Reviewing files that changed from the base of the PR and between 2166de4 and 2a2fda7.

📒 Files selected for processing (10)

• internal/cmn/schema/dag.schema.json
• internal/core/spec/dag.go
• internal/core/spec/llm_test.go
• internal/core/spec/spec003_conformance_test.go
• internal/core/spec/step.go
• internal/core/value_fields.go
• internal/core/value_fields_test.go
• internal/runtime/builtin/chat/executor.go
• internal/runtime/builtin/chat/executor_test.go
• specs/003-value-resolution.md

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.