docs(agents): verify and refresh AGENTS.md + CLAUDE.md against the current codebase by Gajesh2007 · Pull Request #495 · Layr-Labs/d-inference

Gajesh2007 · 2026-07-02T05:35:57Z

Summary

Full verification pass over both agent-onboarding docs: every factual claim treated as a hypothesis and checked against source. ~96 insertions / 56 deletions across the two files.

Before / After

flowchart LR
  subgraph Before
    A1["agent reads AGENTS.md/CLAUDE.md"] --> B1["stale facts: 3x memory rule (gone),<br/>ScoreProvider (deleted), ChatML inject (never true),<br/>Solana billing (removed), wrong package descriptions,<br/>wrong paths, missing subsystems"]
    B1 --> C1["agent designs against a codebase<br/>that no longer exists"]
  end
  subgraph After
    A2["agent reads AGENTS.md/CLAUDE.md"] --> B2["UnifiedMemoryCap model, cost-based selection,<br/>TemplateRenderCheck reality, current routes/packages,<br/>new hot-path subsystems documented,<br/>hypervisor legacy-compat note"]
    B2 --> C2["docs match code<br/>(every cited path/symbol existence-checked)"]
  end

Highlights

Memory model: replaced the dead estimatedMemoryGb × 3.0 rule with UnifiedMemoryCap (0.90 cap, 4 GiB load headroom, post-load KV guard, env overrides) and the freeForLoadGB mirroring nuance
Scheduler: ScoreProvider (deleted code) removed from both files; cost/penalty model documented; pendingModelLoads reader list corrected
False claims removed: ChatML auto-injection (no such code — replaced with TemplateRenderCheck/template_render_ok routing), Solana billing rail, CLAUDE.md's "Postgres not used in production yet"
Paths/structure: coordinator/internal/e2e/, telemetry/ vs datadog/ descriptions, consumer route list, CLI command list, runbook paths, admin-ui/landing/deploy + 7 missing coordinator packages added to the maps
New subsystems documented (one bullet each): single-parse frame decode, chunk-key cache lifecycle, two-lane writer + watchdog, overflow grace-then-499 policy, hypervisor legacy-compat (chore(attestation): retire the hypervisor_active stub — field, entitlement, and false UI claims #492)
Full verified-accurate list, evidence per correction, and overlap assessment (~60-65% duplication between the two files) in the agent report

Open questions for reviewers

CLAUDE.md Infrastructure table says the prod console is an EigenCloud app — unverifiable from the repo (dev is Vercel). Left as-is; please confirm or correct.
The continuous-batching benchmark figures (B=4 speedups) live in the submodule and were left unverified.
Out of scope but flagged: coordinator/registry/scheduler.go:60 comment still references the deleted scripts/calibrate-routing.sh.

^{Need help on this PR? Tag /codesmith with what you need. Autofix is disabled.}

…rrent codebase Every factual claim in both files was checked against source; stale ones fixed with evidence, missing subsystems added, deleted things purged. Headline corrections: - Memory model: the 'estimatedMemoryGb * 3.0 headroom' rule is long gone — documented UnifiedMemoryCap (0.90 cap fraction, 2 GiB OS floor, 4 GiB load headroom, post-load measured-KV guard, env overrides) and the freeForLoadGB mirroring on the coordinator side. - Provider scoring: ScoreProvider no longer exists; selection is cost/penalty-based. WarmModels warm-bonus claim removed. - pendingModelLoads readers: now also cold-spill eligibility + warm-pool controller. - coordinator/e2e/ -> coordinator/internal/e2e/; telemetry/ vs datadog/ package descriptions swapped to reality; consumer routes (no transcriptions/images; /v1/responses exists); Solana rail removed from billing descriptions; darkbloom CLI command list (no serve). - ChatML auto-inject claim was false — replaced with TemplateRenderCheck / template_render_ok routing reality. - LatestProviderVersion is the no-release-row fallback, not the source of truth; release tag shapes; runbook paths (docs/operations/*). - CLAUDE.md store-selection claim ('Postgres not used in production yet') was flat wrong; ARV enforcement claim softened to logged-only. - 12 -> 14 E2E tests; structure maps gain apns/config/env/profilesign/ stateexport/routingsim/api/types, admin-ui/, landing/, deploy/. Added (one bullet each): type_scan single-parse decode, chunk_key_cache forget-on-terminal lifecycle, two-lane provider writer + watchdog, chunk-overflow 250ms-grace-then-499 policy, hypervisor_active legacy-compat note (#492). Open question flagged for review: CLAUDE.md Infrastructure table claims the prod console is an EigenCloud app — could not be confirmed from the repo; left unchanged.

vercel · 2026-07-02T05:35:59Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
d-inference	Ready	Preview	Jul 2, 2026 11:58pm
d-inference-console-ui-dev	Ready	Preview	Jul 2, 2026 11:58pm
d-inference-landing	Ready	Preview	Jul 2, 2026 11:58pm

github-actions · 2026-07-02T05:36:23Z

No security-relevant changes: this PR only modifies AI agent instruction files and has no impact on any trust boundary or asset defined in the threat model.

Neither AGENTS.md nor CLAUDE.md is present in any affected_files list, and neither file is part of the production runtime (coordinator, provider-swift, or console-ui). They contain no secrets, no auth logic, no network handlers, and no build artifacts that ship to production.

New trust-boundary surface? No. These files are developer-tooling documentation. They do not introduce new API surface, credentials, or trust relationships. No additions to the threat model are warranted.

One housekeeping note: if either file is ever updated to include real secrets (e.g., example API keys, coordinator URLs pointing to production), that would be an out-of-band credential exposure — but the current content does not do this.

🔐 Threat model: docs/threat-model.yaml · Updates on each push to this PR

ethenotethan

Automated Code Review — Layr-Labs/d-inference#

Verdict: COMMENT

Security — ✅ No issues found

Performance — ✅ No issues found

Type_diligence — ✅ No issues found

Additive_complexity — ✅ No issues found

✅ All four passes clean. No issues found.

🤖 Automated review by Centaur · DAR-186

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5e33baaf58

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-07-02T05:46:45Z

+(git submodules; a third submodule, `libs/mlx`, provides the MLX source that
+`scripts/fetch-metallib.sh` builds the metallib from).


Point metallib guidance at the actual source tree

When a developer or release agent needs to update/debug Metal kernels, this sends them to the top-level libs/mlx submodule, but the build path does not use it: scripts/fetch-metallib.sh:24-25 sets MLX_SRC to libs/mlx-swift/Source/Cmlx/mlx, and the release workflow uses the same path at .github/workflows/release-swift.yml:217-230. Updating libs/mlx based on this guidance would not change the generated mlx.metallib, risking a release with kernels still tied to the nested MLX source.

Useful? React with 👍 / 👎.

Fixed in 9940af5 — the doc now points at libs/mlx-swift/Source/Cmlx/mlx (what fetch-metallib.sh:25 and the release workflow actually build from) and explicitly warns that updating top-level libs/mlx alone does not change the generated kernels.

chatgpt-codex-connector · 2026-07-02T05:46:45Z

 - Provider bundle creation (staging, .app wrapping, signing, notarization) lives inline in `.github/workflows/release-swift.yml` (bundle steps ~341-617); there is no standalone bundling script.
 - Installer flow lives in `scripts/install.sh`.
- Provider update checks use `LatestProviderVersion` in `coordinator/api/server.go`, so bundle uploads and version bumps need to stay coordinated.
+- Provider update checks read the latest registered release from the store (CI registers via `POST /v1/releases`); `LatestProviderVersion` in `coordinator/api/server.go` is only the fallback when no release row exists and must stay in sync with `ProviderCore.version`, so bundle uploads and version bumps need to stay coordinated.


Don't imply LatestProviderVersion backs self-updates

This says the hardcoded version is the fallback when no release row exists, but that only applies to /api/version; the actual installer and darkbloom update path read /v1/releases/latest, whose handler returns 404 when GetLatestRelease is nil. In a dev/in-memory or mis-registered release scenario, keeping LatestProviderVersion synced will not make installs or self-updates work, so this guidance can cause release fixes to miss the required release-row registration.

Useful? React with 👍 / 👎.

Fixed in 9940af5 — the bullet now leads with the real dependency: GET /v1/releases/latest 404s without a release row, so installs/self-updates are fixed by registering the release; LatestProviderVersion is described as the version-display fallback only.

chatgpt-codex-connector · 2026-07-02T05:46:45Z

- Model scan uses fast discovery (no hashing) at startup. Weight hashing is on-demand via `compute_weight_hash()` only for the served model. Don't add hashing back to the scan path.
- Provider auto-injects ChatML template for models missing `chat_template` field. This is intentional -- Qwen3.5 base models ship without it.
+- Model scan uses fast discovery (no hashing) at startup (`ModelScanner`). Weight hashing is on-demand via `WeightHasher.computeHash(for:)` only for models that need attestation/verification. Don't add hashing back to the scan path.
+- Models with broken chat templates are not auto-repaired. The provider runs a scan-time chat-template render self-check (`TemplateRenderCheck`) and reports `template_render_ok=false`; the coordinator then keeps tool-bearing requests away from those models.


Describe template_render_ok as an all-request gate

The current routing gate rejects every request for a provider/model with an explicit template_render_ok=false: providerEligibleForTraitsLocked returns false before applying trait-specific floors. This new wording says the coordinator keeps only tool-bearing requests away, which can mislead future routing changes or incident triage into allowing plain text traffic to a model whose chat template self-check failed.

Useful? React with 👍 / 👎.

Fixed in 9940af5 — now states the template_render_ok=false gate fences ALL request shapes for the (provider, model) pair (matching providerEligibleForTraitsLocked's own doc), with only the capability version floors being tool-scoped.

- metallib source is libs/mlx-swift/Source/Cmlx/mlx (the nested tree Cmlx compiles against), not the top-level libs/mlx submodule - /v1/releases/latest 404s without a release row: installs/self-updates are fixed by registering the release; LatestProviderVersion only backs the version-display fallback - template_render_ok=false fences ALL request shapes, not just tool-bearing ones; only capability version floors are tool-scoped

ethenotethan

Automated Code Review — Layr-Labs/d-inference#

Verdict: COMMENT

Security — ✅ No issues found

Performance — ✅ No issues found

Type_diligence — ✅ No issues found

Additive_complexity — ✅ No issues found

✅ All four passes clean. No issues found.

🤖 Automated review by Centaur · DAR-186

blacksmith-sh · 2026-07-03T00:05:25Z

Found 1 test failure on Blacksmith runners:

Failure

Test	View Logs
`github.com/eigeninference/d-inference/e2e/TestIntegration_ConcurrentRequests`	View Logs

^{Need help on this PR? Tag /codesmith with what you need.}

ethenotethan

Automated Code Review — Layr-Labs/d-inference#

Verdict: COMMENT

Security — ✅ No issues found

Performance — ✅ No issues found

Type_diligence — ✅ No issues found

Additive_complexity — ✅ No issues found

✅ All four passes clean. No issues found.

🤖 Automated review by Centaur · DAR-186

Gajesh2007 requested a deployment to benchmarks July 2, 2026 05:36 — with GitHub Actions Waiting

vercel Bot deployed to Preview – d-inference-landing July 2, 2026 05:36 View deployment

vercel Bot deployed to Preview – d-inference July 2, 2026 05:36 View deployment

vercel Bot deployed to Preview – d-inference-console-ui-dev July 2, 2026 05:36 View deployment

ethenotethan reviewed Jul 2, 2026

View reviewed changes

chatgpt-codex-connector Bot reviewed Jul 2, 2026

View reviewed changes

Gajesh2007 requested a deployment to benchmarks July 2, 2026 06:08 — with GitHub Actions Waiting

vercel Bot deployed to Preview – d-inference-landing July 2, 2026 06:08 View deployment

vercel Bot deployed to Preview – d-inference July 2, 2026 06:09 View deployment

vercel Bot deployed to Preview – d-inference-console-ui-dev July 2, 2026 06:09 View deployment

ethenotethan reviewed Jul 2, 2026

View reviewed changes

Merge branch 'master' into docs/agents-md-refresh

7fa2fac

Gajesh2007 requested a deployment to benchmarks July 2, 2026 23:58 — with GitHub Actions Waiting

vercel Bot deployed to Preview – d-inference-landing July 2, 2026 23:58 View deployment

vercel Bot deployed to Preview – d-inference July 2, 2026 23:58 View deployment

vercel Bot deployed to Preview – d-inference-console-ui-dev July 2, 2026 23:58 View deployment

ethenotethan reviewed Jul 3, 2026

View reviewed changes

Gajesh2007 merged commit 9920498 into master Jul 3, 2026
15 of 17 checks passed

		(git submodules; a third submodule, `libs/mlx`, provides the MLX source that
		`scripts/fetch-metallib.sh` builds the metallib from).

Uh oh!

Conversation

Gajesh2007 commented Jul 2, 2026 • edited by blacksmith-sh Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Before / After

Highlights

Open questions for reviewers

Uh oh!

vercel Bot commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ethenotethan left a comment

Choose a reason for hiding this comment

Automated Code Review — Layr-Labs/d-inference#

Security — ✅ No issues found

Performance — ✅ No issues found

Type_diligence — ✅ No issues found

Additive_complexity — ✅ No issues found

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jul 2, 2026

Choose a reason for hiding this comment

Uh oh!

Gajesh2007 Jul 2, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 2, 2026

Choose a reason for hiding this comment

Uh oh!

Gajesh2007 Jul 2, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 2, 2026

Choose a reason for hiding this comment

Uh oh!

Gajesh2007 Jul 2, 2026

Choose a reason for hiding this comment

Uh oh!

ethenotethan left a comment

Choose a reason for hiding this comment

Automated Code Review — Layr-Labs/d-inference#

Security — ✅ No issues found

Performance — ✅ No issues found

Type_diligence — ✅ No issues found

Additive_complexity — ✅ No issues found

Uh oh!

blacksmith-sh Bot commented Jul 3, 2026

Failure

Uh oh!

ethenotethan left a comment

Choose a reason for hiding this comment

Automated Code Review — Layr-Labs/d-inference#

Security — ✅ No issues found

Performance — ✅ No issues found

Type_diligence — ✅ No issues found

Additive_complexity — ✅ No issues found

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Gajesh2007 commented Jul 2, 2026 •

edited by blacksmith-sh Bot

Loading

vercel Bot commented Jul 2, 2026 •

edited

Loading

github-actions Bot commented Jul 2, 2026 •

edited

Loading