Skip to content

[fix] Make credential resolution work for provider keys, bare models, and Bedrock#5057

Merged
junaway merged 5 commits into
big-agentsfrom
fix/providers-models-secrets-review-part-1
Jul 3, 2026
Merged

[fix] Make credential resolution work for provider keys, bare models, and Bedrock#5057
junaway merged 5 commits into
big-agentsfrom
fix/providers-models-secrets-review-part-1

Conversation

@jp-agenta

@jp-agenta jp-agenta commented Jul 3, 2026

Copy link
Copy Markdown
Member

Context

Three failures around model credentials, found by walking one scenario end to end (a fresh agent, then a Bedrock connection):

  1. A brand-new agent from the default template could not run. The default was llm: {model: "gpt-5.5"} with no provider, and the resolver refuses bare model ids (F-017): model 'gpt-5.5' needs a provider prefix.
  2. Adding an OpenAI key did not help. The resolver required a slug on standard provider keys (from header.name, which standard keys never have), so every plain provider key was invisible: no connection resolved for provider 'openai'.
  3. A Bedrock connection was unusable end to end: the drawer had no field for the Bedrock API key (bearer token), the model picker never showed a custom connection's models, and picking one wrote provider: "bedrock", which the harness check rejects (provider 'bedrock' is not supported by harness 'claude') because bedrock is a deployment, not a provider family.

Changes

Default template carries a provider. build_agent_v0_default now emits llm: {provider: "openai", model: ...}. The provider constant derives from the default harness's reachable set, so the two defaults cannot drift apart.

Provider keys resolve by provider, not by name. _provider_key_candidate no longer requires header.name. A standard key is identified by its provider family; header.name was never a slug.

One provider inference for bare ids. infer_provider_from(model) merges the Claude alias set, the claude-* structural rule, and an inverted supported_llm_models catalog (unambiguous ids only). Known bare ids like gpt-5.5 resolve; unknown or cross-provider ids still fail loud.

Bedrock, end to end.

  • The drawer gets a "Bedrock API key" field. Auth requirements are declarative (PROVIDER_AUTH_REQUIREMENTS): bearer token OR access-key pair, validated as a group instead of hardcoded per-field checks.
  • The stored token reaches both harnesses as AWS_BEARER_TOKEN_BEDROCK (the one channel Pi and Claude Code both accept), and no longer leaks into ANTHROPIC_API_KEY.
  • The agent model picker now lists each vault connection's own models, gated by the harness's consumable deployments (bedrock shows under Claude Code, not under Pi). Picking one writes the model family as the provider (familyFromModelId: eu.anthropic.claude-haiku-4-5 gives anthropic) and auto-fills the connection slug.

Before, picking the Bedrock model produced a config that could not pass the harness gate:

{"model": "eu.anthropic.claude-haiku-4-5", "provider": "bedrock"}

After:

{"model": "eu.anthropic.claude-haiku-4-5", "provider": "anthropic",
 "connection": {"mode": "agenta", "slug": "my-bedrock"}}

The resolver then injects AWS_BEARER_TOKEN_BEDROCK + AWS_REGION, and the runner sets CLAUDE_CODE_USE_BEDROCK=1.

Commit modal. The "Model & harness" diff section now shows llm.model and llm.provider rows (provider moved out of Advanced).

Runner. One log line per run states the resolved model, provider, deployment, connection, and secret key names (never values). This is what located every failure above.

Tests / notes

  • SDK: 50/50 connection unit tests; resolution verified live for a plain OpenAI key, a bare model id, and a Bedrock connection (env contains bearer + region only, both harness gates pass).
  • Web: tsc --noEmit clean on entity-ui/entities/shared; eslint clean.
  • Runner: tsc --noEmit clean, vitest green.
  • Verified on the dev stack: a Claude Code + Bedrock run completes in ~6s with the injected bearer token.
  • gpt-5.5 as the default model id is a separate data question (it resolves and authenticates, but is not a real OpenAI id).

What to QA

  • Create a new agent from the template and run it with only an OpenAI provider key in the vault. It answers; no "needs a provider prefix" error.
  • Add a custom Bedrock provider: the drawer shows "Bedrock API key" with an either/or hint; submitting with neither a bearer token nor an access-key pair is rejected; region is required.
  • Switch the agent's harness to Claude Code: the model dropdown shows the Bedrock connection's models in their own group. Pick one, save, run. Under Pi the group is absent.
  • Regression: the completion playground's model picker and an existing OpenAI-based agent still resolve and run unchanged.

@dosubot dosubot Bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Jul 3, 2026
@vercel

vercel Bot commented Jul 3, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
agenta-documentation Ready Ready Preview, Comment Jul 3, 2026 4:00pm

Request Review

@dosubot dosubot Bot added the Bug Report Something isn't working label Jul 3, 2026
@coderabbitai

coderabbitai Bot commented Jul 3, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: a2b5062a-9f48-4b02-88d1-b10e8b3e4824

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/providers-models-secrets-review-part-1

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

…odels-secrets-review-part-1

# Conflicts:
#	web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/useModelHarness.tsx
@junaway junaway changed the title [fix] Resolve some providers/models/secrets issues [fix] Make credential resolution work for provider keys, bare models, and Bedrock Jul 3, 2026
…log lag

The provider/secrets resolver changes moved four things the tests still
encoded the old way, and the OpenRouter model check fought the catalog.

- connections: provider keys are addressed by their PROVIDER (header.name is
  display-only, never a slug), and a bare model id present in the catalog
  infers its provider instead of failing loud. Tests updated + a new
  test_bare_catalog_model_infers_provider.
- default template: the runtime selection is provider-qualified, so the
  /inspect default parses to `provider/model`, not a bare `model`.
- commit diff: agent templates key the model as `llm.model`; the summary
  builder now reads that alongside the legacy `model`. Tests updated.
- supported_llm_models: OpenRouter's list intentionally tracks current
  top-used ids that the pinned litellm build hasn't indexed yet. For that
  provider a miss is expected lag, so it xfails (still catching a typo'd
  prefix structurally) instead of failing CI. The catalog is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@junaway junaway mentioned this pull request Jul 3, 2026
12 tasks
@junaway junaway merged commit 5d1b24a into big-agents Jul 3, 2026
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Bug Report Something isn't working size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants