Skip to content

feat(catalog): persist and reconcile provider snapshots - #905

Merged
chuks-qua merged 7 commits into
mainfrom
feat/893-provider-catalog-snapshots
Jul 20, 2026
Merged

chuks-qua merged 7 commits into
mainfrom
feat/893-provider-catalog-snapshots

Conversation

@chuks-qua

@chuks-qua chuks-qua commented Jul 20, 2026 •

Copy link
Copy Markdown
Contributor

What

Persist provider catalog snapshots per workspace and worktree context, then reconcile background refreshes by stable identity.

Why

Catalog pickers should render the last confirmed capabilities immediately across restarts and provider process eviction without blocking on Codex startup.

Key Changes

  • Store bounded, validated catalog snapshots in SQLite and serve them as stale while refresh runs.
  • Broadcast identity-based entry, agent, diagnostic, and freshness changes while retaining cached state on failures.
  • Preserve logical thread contexts across shared physical refreshes and serialize WebSocket reconciliation safely.
  • Bound refresh work and report explicit overload diagnostics with inactive-context recovery.
  • Add persistence, restart, failure, lifecycle, overflow, and frontend ordering coverage.

Closes #893


View with Codesmith Autofix with Codesmith
Need help on this PR? Tag /codesmith with what you need. Autofix is disabled.

Summary by CodeRabbit

  • New Features
    • Provider catalogs now load quickly from saved snapshots while refreshing in the background.
    • Catalog updates are delivered incrementally, including additions, changes, and removals.
    • Catalog data persists across server restarts and supports workspace and working-directory contexts.
    • The web app applies live catalog updates without overwriting newer information.
  • Bug Fixes
    • Improved handling of unavailable providers, stale data, concurrent refreshes, and connection restarts.
    • Prevented outdated catalog responses from replacing newer updates.

@coderabbitai

coderabbitai Bot commented Jul 20, 2026 •

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@chuks-qua, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 32 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 5509e169-e1a7-42f1-be38-921fd91ba3b5

📥 Commits

Reviewing files that changed from the base of the PR and between 33bc776 and f3733a7.

📒 Files selected for processing (10)
  • apps/server/drizzle/0023_grey_wildside.sql
  • apps/server/drizzle/meta/0023_snapshot.json
  • apps/server/src/repositories/provider-catalog-snapshot-repo.test.ts
  • apps/server/src/repositories/provider-catalog-snapshot-repo.ts
  • apps/server/src/services/provider-catalog-service.test.ts
  • apps/server/src/services/provider-catalog-service.ts
  • apps/server/src/store/schema.ts
  • apps/server/src/transport/ws-router.validation.test.ts
  • apps/web/src/transport/ws-events.test.ts
  • apps/web/src/transport/ws-events.ts
📝 Walkthrough

Walkthrough

Adds persisted provider catalog snapshots with stale-while-revalidate refreshes, bounded reconciliation, server push events, Codex lifecycle handling, and web-store application of incremental catalog changes.

Changes

Provider catalog contracts and persistence

Layer / File(s) Summary
Catalog contracts and snapshot persistence
packages/contracts/src/providers/capability-catalog.ts, packages/contracts/src/index.ts, packages/contracts/src/ws/channels.ts, apps/server/src/store/schema.ts, apps/server/drizzle/*, apps/server/src/repositories/*
Defines catalog change schemas, registers provider.catalogChanged, adds provider_catalog_snapshots, and implements validated SQLite persistence with retention pruning.

Refresh orchestration

Layer / File(s) Summary
Background refresh and reconciliation service
apps/server/src/services/provider-catalog-service.ts, apps/server/src/services/codex-catalog-service.ts, apps/server/src/services/__tests__/*, apps/server/src/services/provider-catalog-service.test.ts
Returns stale snapshots immediately, coordinates bounded refreshes, persists successful or fallback snapshots, emits incremental changes, and preserves Codex snapshots across connection eviction.

Server integration

Layer / File(s) Summary
Server catalog routing and event propagation
apps/server/src/container.ts, apps/server/src/index.ts, apps/server/src/transport/ws-router.ts, apps/server/src/transport/ws-router.validation.test.ts
Wires the repository and service, routes provider.catalog through the service, and broadcasts reconciliation events through WebSocket and IPC paths.

Web client reconciliation

Layer / File(s) Summary
WebSocket change delivery and store reconciliation
apps/web/src/transport/ws-events.ts, apps/web/src/stores/providerCatalogStore.ts, apps/web/src/stores/providerCatalogStore.test.ts
Delivers provider.catalogChanged events to the store, applies identity-based updates around in-flight loads, and reloads after pending-change overflow.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related PRs

  • Mzeey-Empire/mcode#904: Both changes modify the Codex provider.catalog flow and its server integration points.

Poem

A rabbit found snapshots tucked under a tree,
With stale little catalogs waiting patiently.
Refreshes ran softly, diffs hopped in a stream,
The web store kept every unaltered dream.
“Pawsome persistence!” the rabbit cried bright—
“Fresh provider changes now travel just right!”

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 17.65% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly matches the main change: persisting and reconciling provider catalog snapshots.
Description check ✅ Passed The description follows the template's What/Why/Key Changes sections and is detailed enough, with no config changes required.
Linked Issues check ✅ Passed The changes appear to satisfy the linked issue by adding snapshot persistence, stale-while-revalidate refreshes, incremental reconciliation, and the required tests.
Out of Scope Changes check ✅ Passed The diff stays focused on provider catalog persistence, reconciliation, transport wiring, and tests, with no obvious unrelated changes.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
apps/server/src/store/schema.ts (1)

325-341: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick win

Deleted workspaces leave orphaned catalog snapshot rows.

workspaceId has no FK/cascade to workspaces, and the repo's eviction (per the referenced provider-catalog-snapshot-repo.ts upsert) only caps rows per-provider, not per-workspace. Workspace deletion (workspace.delete/workspace.forceDelete in ws-router.ts) doesn't appear to clean up this table, so a deleted workspace's cached snapshot (with its cwd/path) can persist indefinitely until pushed out by unrelated provider activity.

Consider adding a references(() => workspaces.id, { onDelete: "cascade" }) on workspaceId, or an explicit repo method invoked from the workspace delete handlers.

♻️ Proposed schema fix
     contextKey: text("context_key").primaryKey().notNull(),
     providerId: text("provider_id").notNull(),
-    workspaceId: text("workspace_id"),
+    workspaceId: text("workspace_id").references(() => workspaces.id, { onDelete: "cascade" }),
     cwd: text("cwd"),
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/server/src/store/schema.ts` around lines 325 - 341, Update
providerCatalogSnapshots.workspaceId to reference workspaces.id with onDelete
cascade, using the existing workspaces table symbol and preserving its nullable
behavior for non-workspace snapshots. Ensure the schema relation causes catalog
snapshot rows to be removed automatically when a workspace is deleted.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/web/src/transport/ws-events.ts`:
- Around line 348-352: Update the provider.catalogChanged handler in the
pushEmitter subscription to validate each raw payload with
ProviderCatalogChangeSchema before calling
useProviderCatalogStore.getState().reconcile. Only reconcile successfully parsed
ProviderCatalogChange values, and silently drop invalid events so malformed data
cannot reach applyCatalogChange during pending-change replay.

---

Nitpick comments:
In `@apps/server/src/store/schema.ts`:
- Around line 325-341: Update providerCatalogSnapshots.workspaceId to reference
workspaces.id with onDelete cascade, using the existing workspaces table symbol
and preserving its nullable behavior for non-workspace snapshots. Ensure the
schema relation causes catalog snapshot rows to be removed automatically when a
workspace is deleted.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 5ae26e89-33f8-4e8c-af19-e742fd9d5cc1

📥 Commits

Reviewing files that changed from the base of the PR and between 4fdcabf and 33bc776.

📒 Files selected for processing (20)
  • apps/server/drizzle/0023_grey_wildside.sql
  • apps/server/drizzle/meta/0023_snapshot.json
  • apps/server/drizzle/meta/_journal.json
  • apps/server/src/container.ts
  • apps/server/src/index.ts
  • apps/server/src/repositories/provider-catalog-snapshot-repo.test.ts
  • apps/server/src/repositories/provider-catalog-snapshot-repo.ts
  • apps/server/src/services/__tests__/codex-catalog-service.test.ts
  • apps/server/src/services/codex-catalog-service.ts
  • apps/server/src/services/provider-catalog-service.test.ts
  • apps/server/src/services/provider-catalog-service.ts
  • apps/server/src/store/schema.ts
  • apps/server/src/transport/ws-router.ts
  • apps/server/src/transport/ws-router.validation.test.ts
  • apps/web/src/stores/providerCatalogStore.test.ts
  • apps/web/src/stores/providerCatalogStore.ts
  • apps/web/src/transport/ws-events.ts
  • packages/contracts/src/index.ts
  • packages/contracts/src/providers/capability-catalog.ts
  • packages/contracts/src/ws/channels.ts

Comment thread apps/web/src/transport/ws-events.ts
@chuks-qua
chuks-qua merged commit 5c6a1c8 into main Jul 20, 2026
10 checks passed
@chuks-qua
chuks-qua deleted the feat/893-provider-catalog-snapshots branch July 20, 2026 21:47
@mze-bot mze-bot Bot mentioned this pull request Aug 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: persist and reconcile provider catalog snapshots

1 participant