Convert Lead on the hosted 4.0.1 environment jack fails with "Record <id> not found in crm_lead"; it does not reproduce on 17.7.0 locally

Filing gate: ① a product defect with reach measured. The maintainer measured it on a production environment. Filed by the repo:hotcrm seat, session_012zh91QzFgePbkmuHnugLN3.

Who acts on it: the maintainer first, for one reading from jack that only they can take (the Maintainer-action: line below). Once the failing request is known, the fix is routed to its owning repo: hotcrm if it is the app's metadata or data, objectstack or cloud if it is the runtime.

What was seen

On jack.objectos.ai (HotCRM 4.0.1, upgraded in place from 3.1.0 on 2026-10-11), the maintainer clicked Convert Lead on lead G672wEFPGyEphIW3 (/_console/apps/app.objectstack.hotcrm/crm_lead/record/G672wEFPGyEphIW3). The console answered:

Record G672wEFPGyEphIW3 not found in crm_lead

The record page itself opens, so the lead exists and the user can read it.

Where that message comes from

It is recordNotFoundError (@objectstack/core, utils/record-not-found.ts), code RECORD_NOT_FOUND, status 404.
It is thrown by the data protocol (metadata-protocol) when engine.findOne returns no row under the caller's context. Three verbs throw it: getData, cloneData and updateData.
It reads as a REST data call, not a flow node. A failing node in this flow answers Node '<id>' failed: …; the browser pass measured that shape in A-01 (Browser pass #2058 — HotCRM-side defects (18 items: accepted quote still takes lines, half-converted leads, agents losing their cases, unreachable Sales Home, …) — list card, split per item at dispatch #2060 item 2).

Local measurement: does not reproduce

Run on HotCRM main 1d7148bf (4.0.1), @objectstack/* 17.7.0, a fresh dev DB, signed in as the dev admin, in Chromium.

Lead	Organization	Create Opportunity	Result
`59ziddzSX-Ym-8l4` (seeded)	`organization_id: null`	no	converted, account and contact set
`7BUtVE1jhoMEoQFB` (seeded)	`organization_id: null`	yes, 50000	converted, account, contact and opportunity set

In both runs, POST /automation/lead_conversion/trigger answered 200 (paused at screen_1), and POST …/runs/<id>/resume answered 200 with failed: 0.
No /api/ call returned 4xx.
Seeded leads carrying organization_id: null (A-16) made no difference here.

What is left: something only the hosted environment has. Three candidates, none measured yet:

the runtime cloud serves (its framework pin, not 17.7.0);
the console build hosted serves;
rows upgraded from 3.1.0. The hosted schema sync is additive, and 4.0 added lead columns with no backfill. 4.0 upgrade on a hosted environment: a contact's mailing address disappears from the record page, and a hosted customer has no way to run the documented conversion #2062 is the same mechanism on contacts.

First act

Maintainer-action: on jack, open DevTools → Network, click Convert Lead on G672wEFPGyEphIW3 again, and record the failing request's method, URL and response body here. Also record whether the lead afterwards reads converted, and whether an account or contact was created. — done when a comment on this card names the failing request and its response

With that reading, the seat routes the fix (or files it upstream) and re-measures.

Duplicate check

Every hotcrm issue and PR was listed (/issues?state=all, the 1,896-row index plus everything updated since 2026-10-09).
They were matched case-insensitively on not found in crm_lead, RECORD_NOT_FOUND, Record … not found, and conversion on a hosted environment. The only hit is fix(profiles, docs): reps read and join the Discussion of records they can read; the expiry and renewal sweeps run where scheduled work is on; the automation page names the shipped templates (#2029 items 3, 5, 6) #2043 (closed, discussion access), which is unrelated.
The nearest open card is Browser pass #2058 — HotCRM-side defects (18 items: accepted quote still takes lines, half-converted leads, agents losing their cases, unreachable Sales Home, …) — list card, split per item at dispatch #2060 item 2 (A-01), which is a different failure: "Amount is required" from create_opportunity.

Generated by Claude Code

Root cause measured: the runtime, not HotCRM. Routed to objectstack-ai/objectstack#22944

The Maintainer-action: reading is no longer needed. The failing request was reproduced locally on a hosted-shape rig whose pins equal cloud v1.36.0, the release now serving jack:

cloud two-process stack, run-stack.sh --prod-like --with-ui
framework 56bf27affb, objectui d758f2f20d
HotCRM 3.1.0 installed from the local marketplace, then upgraded in place to 4.0.1, the path jack took

The failing request (seeded lead GTS2jcGfJc2tHdxh, clicked in the console):

POST /api/v1/actions/crm_lead/convert_lead   {"recordId":"GTS2jcGfJc2tHdxh","params":{}}
→ 404 {"code":"RECORD_NOT_FOUND","message":"Record GTS2jcGfJc2tHdxh not found in crm_lead"}

The same session reads GET /api/v1/data/crm_lead/GTS2jcGfJc2tHdxh → 200.

Why:

objectui#12046 (in cloud v1.36.0, deployed 2026-10-11) sends a declared flow action's click through the action door instead of /automation/:flow/trigger.
On cloud's multi-tenant host, that door resolves protocol on the host kernel before the environment's own kernel. The host's protocol has no crm_lead, so it throws OBJECT_NOT_FOUND.
The subject-record load swallows that error, and the door answers RECORD_NOT_FOUND.

Full mechanism and probe output are on objectstack-ai/objectstack#22944.

Why the earlier local run did not reproduce it:

It ran standalone, with one kernel.
It ran console 17.7.0, which still used the trigger route. On this rig the trigger route answers 200 too.

What it means for HotCRM: nothing in the app's metadata or its upgraded 3.1.0 rows is involved. The same 404 hits:

the flow schedule_followup;
the script mark_article_helpful;
by construction, every record-bound action in 4.0.1 (23 script, 7 flow) on every hosted environment.

Verified fix direction: resolving on the request's own kernel first was patched into the rig's runtime build. With it:

Convert Lead completes in the console: lead converted, account, contact and opportunity (50000) created.
The other actions sampled answer 200.
A nonexistent id still answers 404.

Status: this card stays OPEN. It is blocked on objectstack-ai/objectstack#22944 landing and on cloud's framework pin carrying it. Done when Convert Lead succeeds on jack after that deploy.

🤖 Generated with Claude Code

How this ships (maintainer decision, 2026-10-12): the fix ships with cloud's v18 framework-pin move, objectstack-ai/cloud#2709. There is no hotfix pin and no console rollback. The fix itself is PR objectstack-ai/objectstack#22945 (card objectstack-ai/objectstack#22944).

Until that move deploys, Convert Lead and every other record action keep answering 404 on hosted environments, and nothing on the HotCRM side can change that. This card stays OPEN, blocked. It is done when Convert Lead succeeds on jack after the cloud#2709 deploy.

🤖 Generated with Claude Code

State bookkeeping by the repo:hotcrm seat, session_012zh91QzFgePbkmuHnugLN3, 2026-10-12T06:29Z.

The maintainer's two comments above answer the Maintainer-action: line in the body, and move this card to pm:blocked.
The unlock scan reads only a line-start Blocked-by: line, and this card had none. It is added below.
It points at the deploy vehicle the maintainer named, not at the fix card. #22944 closes when PR #22945 merges, but this card is done only when the fix is deployed.

Blocked-by: objectstack-ai/cloud#2709

The fix: runtime: on a kernel-resolver host (cloud), every record-bound POST /actions answers 404 RECORD_NOT_FOUND — resolveService asks the host kernel first, and its singleton protocol has none of the environment's objects objectstack#22944, landed by PR fix(runtime): on a kernel-resolver host, resolve a service on the request's own kernel first — record-bound POST /actions stops answering 404 RECORD_NOT_FOUND objectstack#22945.
Done-when stays the maintainer's: Convert Lead succeeds on jack after the cloud#2709 deploy.
Nothing to dispatch on the HotCRM side. The seat re-reads this card on each wake.

Generated by Claude Code

Navigation Menu

Repository navigation

Convert Lead on the hosted 4.0.1 environment `jack` fails with "Record <id> not found in crm_lead"; it does not reproduce on 17.7.0 locally #2065

What was seen

Where that message comes from

Local measurement: does not reproduce

First act

Duplicate check

Root cause measured: the runtime, not HotCRM. Routed to objectstack-ai/objectstack#22944

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Convert Lead on the hosted 4.0.1 environment jack fails with "Record <id> not found in crm_lead"; it does not reproduce on 17.7.0 locally #2065

Description

What was seen

Where that message comes from

Local measurement: does not reproduce

First act

Duplicate check

Activity

hotlong commented on Oct 12, 2026

Root cause measured: the runtime, not HotCRM. Routed to objectstack-ai/objectstack#22944

hotlong commented on Oct 12, 2026

objectstack-fleet commented on Oct 12, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Convert Lead on the hosted 4.0.1 environment `jack` fails with "Record <id> not found in crm_lead"; it does not reproduce on 17.7.0 locally #2065