feat(backend): add idempotent LLM catalog importer and bundled catalog

[ntindle]

Why

Part 3 of 9 of the LLM registry restack (#13605 → #13606 → this). The original stack seeded the registry with a one-shot SQL migration — frozen at install time, no way to update self-hosted installs, and no protection for admin edits. This PR replaces seeding with an idempotent importer that runs on every startup, which is the foundation the remote-sync mechanism (part 5) builds on.

What

• catalog_model.py — CatalogPayload schema v1: the single source of truth for the bundled file, the public endpoint (part 4), and sync validation (part 5). Facts only: no costs, no credentials, no routing cells, no non-GA models. Slug patterns, bounds, and a 2000-model cap.
• importer.py — merge rules: upsert-by-slug; source=LOCAL (admin-owned) rows never updated or disabled, enforced inside every UPDATE's WHERE clause so a concurrent admin claim wins the race; absent models are disabled + catalogRemovedAt-stamped, never deleted; LlmModelCost never touched; sha256 content-hash fast-path (steady-state boot = one SELECT); one transaction, then cache refresh + cross-pod fan-out.
• export.py — regenerates catalog.json from a populated DB (python -m backend.data.llm_registry.export); GA-visibility models only.
• catalog.json — 85 models / 8 providers / 17 creators, generated from today's MODEL_METADATA (deliberately not the March-era seed data); gpt-5.2 recommended, mirroring DEFAULT_LLM_MODEL.
• Lifespan — bundled import before registry refresh, fail-soft (empty registry degrades to hardcoded behavior).

Verification

• 51/51 tests pass locally — real-DB integration tests for each merge rule (create, hash fast-path, LOCAL survival under both update and removal, disable-not-delete, reappearance re-enable, forward-referenced fallback pointers, schema_version rejection, full bundled import), catalog schema validation edges, and a forever-guard that catalog.json parses with unique slugs and sound references
• poetry run format + poetry run lint clean

Checklist

• data/*.py changes: registry tables are global config, no user-scoped rows — no user ID checks applicable
• Out-of-scope changes: none

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 7df1d5c6-9b41-4fa8-87a6-2c3d44283b81

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/llm-registry-v2-importer

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 76.01%. Comparing base (2a3a7f0) to head (47f709d).

Additional details and impacted files

@@                    Coverage Diff                     @@
##           feat/llm-registry-v2-core   #13607   +/-   ##
==========================================================
  Coverage                      76.01%   76.01%           
==========================================================
  Files                           2688     2688           
  Lines                         204280   204280           
  Branches                       19654    19654           
==========================================================
+ Hits                          155278   155281    +3     
+ Misses                         44662    44657    -5     
- Partials                        4340     4342    +2     

Flag	Coverage Δ
platform-frontend-e2e	31.05% <ø> (+0.06%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
Platform Backend	82.83% <ø> (ø)
Platform Frontend	49.98% <ø> (+<0.01%)	⬆️
AutoGPT Libs	∅ <ø> (∅)
Classic AutoGPT	28.43% <ø> (ø)

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[ntindle]

After a post-build design review we pivoted from the DB-registry to catalog-as-code: one canonical, schema-validated catalog file is the source of truth (models + costs + copilot routing), updated by PR (catalog-only diffs may ride hotfix→master; the /review bot covers the lane), propagated by CD, with LaunchDarkly demoted to optional cohort experiments and git history as the audit log. Rationale: the admin machinery mostly bought instant-mutation and non-engineer authoring we don't need, and LD is too unreliable to be the load-bearing control layer.

Replaced by #13622 — catalog.py is promoted from seed data to the database itself; the importer/merge semantics have no second writer to exist for.