feat: durable automatic multi-provider catalog

[seonghobae]

Summary

Build a durable provider control plane for the five organization credentials:

• NVIDIA_NIM_API_KEY
• NVIDIA_NIM_API_KEY_SUB
• BYTEZ_API_KEY
• OPENROUTER_API_KEY
• OPENAI_API_KEY

The feature keeps GitHub Actions secrets as one-shot bootstrap transport, stores provider values only in the existing pgcrypto credential registry, discovers provider models account by account, persists normalized catalog metadata, and constructs the ordinary paper-grounded TaskOrchestrator pool automatically.

Architecture

• NVIDIA primary and secondary credentials remain independent provider accounts so quota, revocation, health, and failover state are not conflated.
• provider_accounts, provider_models, model_capabilities, model_modalities, and catalog_refresh_runs form a third-normal-form catalog. Catalog rows reference credential names and never contain secret values.
• Successful account refreshes atomically replace that account’s current model set. Failed refreshes append stable failure evidence and preserve last-known-good models.
• OpenAI, OpenRouter, and NVIDIA NIM continue through the hardened OpenAI-compatible transport inherited from this branch’s security baseline.
• Bytez uses a narrow native Key/input adapter and fails closed for unsupported passthrough shapes instead of fabricating OpenAI response objects.
• --provider-catalog-dsn makes the durable catalog authoritative at startup. An empty or unavailable catalog is an error; there is no implicit mock-agent fallback.
• Capability/role fit drives Thinker, Worker, Verifier, and Synthesizer selection. Context and known price are bounded tie-breaks; price never substitutes for task fit.

GitHub Actions trust boundary

.github/workflows/provider-catalog-sync.yml separates untrusted and trusted execution:

• pull requests receive no provider or database secrets and run deterministic offline contracts only;
• scheduled/manual execution runs only on protected main in the production environment;
• production sync requires the complete five-key inventory plus CONTEXTUAL_ORCHESTRATOR_KV_DSN and CONTEXTUAL_ORCHESTRATOR_KV_PASSPHRASE;
• missing durable bootstrap configuration fails closed rather than reporting success against process memory;
• generated model-agent evidence is checked against all exact secret values before the job completes.

Actual production credential registration is intentionally not claimed by this PR: GitHub does not expose secret values to PR code. The protected-default-branch sync must run successfully after integration to establish live DB evidence.

Robustness

• bounded attempts, full-jitter backoff, deadlines, HTTPS-only catalog access, public-address validation, direct DNS-pinned connections, standard TLS/SNI verification, redirect rejection, bounded response bodies, stable secret-free error codes;
• provider-local failure isolation and last-known-good service;
• fail-closed authentication/schema/database/empty-catalog handling;
• credential rotation by stable registry name without rewriting model records;
• existing per-agent retry, cross-provider failover, and circuit breaker remain active in TaskOrchestrator.

Verification scope

The branch includes deterministic contracts for fixed inventory, all-or-nothing bootstrap, normalized model metadata, malformed/non-finite values, provider failure isolation, stale catalogs, no-candidate startup, generated agent naming, role routing, cross-provider failover, native Bytez handling, catalog-backed CLI startup, and no-secret 3NF DDL.

Full exact-head repository checks, 100% branch/docstring coverage, security/fuzz scans, current semantic reviews, and protected merge authorization remain required. This PR is stacked on fix/atheris-interpreter-lock so it inherits the accepted provider egress/response security prerequisite rather than duplicating or bypassing it.

Documentation

• design specification and Superpowers implementation plan;
• operator guide;
• APA 7 doctoring with failure, rotation, incident, rollback, and evidence interpretation;
• database design and CHANGELOG updates.

Refs #95 and the provider-catalog follow-up to #96.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: c65a8817-5af7-44cb-9b38-4919ad50d049

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}