Skip to content

Commit 99ccbb9

Browse files
Sam Altmanclaude
andauthored
fix(service-settings): say the AI runtime is cloud-only instead of telling operators to mount it (#11467)
The `ai/test` fallback handler answered all three real-provider branches with "Mount @objectstack/service-ai to exercise live calls", while this platform's own roster — PLATFORM_CAPABILITY_PROVIDERS.ai in @objectstack/spec/kernel — declares edition: 'cloud', defined as "no installable version in the open edition". The instruction is kept and gains the boundary it was missing. The sentence is read from the roster (package + note verbatim) rather than hand-written a fourth time, matching how packages/cli's capability preflight already surfaces the same note. The three provider prefixes stay distinct; only the shared trailing sentence converges. Fixes #11318 Claude-Session: https://claude.ai/code/session_01APWX2AwT3a4xDcjPCe8bk4 Co-authored-by: Claude <noreply@anthropic.com>
1 parent b372318 commit 99ccbb9

3 files changed

Lines changed: 172 additions & 3 deletions

File tree

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
---
2+
'@objectstack/service-settings': patch
3+
---
4+
5+
Tell the operator in Settings → AI that `@objectstack/service-ai` has no
6+
open-edition version to install, instead of three bare "Mount it" lines
7+
8+
Configure any real LLM provider in **Settings → AI** and press *Test
9+
connection*, and the built-in fallback handler answered — on all three
10+
real-provider branches — "Mount `@objectstack/service-ai` to exercise live
11+
calls." This platform's own capability roster says that cannot be done:
12+
`PLATFORM_CAPABILITY_PROVIDERS.ai` in `@objectstack/spec/kernel` declares
13+
`edition: 'cloud'`, which `CapabilityEdition` defines as "realized only by a
14+
cloud runtime tier; there is **no installable version in the open edition**" —
15+
the un-followable "add it to your dependencies" that framework#3366 exists to
16+
make legible. The package is in no directory of this repo (0 path hits for
17+
`/service-ai/` on `main`; `/service-settings/` returns 64 and
18+
`/embedder-openai/` 8 under the identical command, so the zero is real).
19+
20+
The instruction is **kept** — an operator may well have a cloud tier — and
21+
gains the boundary it was missing, so one who does not can see the path is
22+
closed to them.
23+
24+
Worse than a 404, the install succeeds. Measured 2026-08-23 against the public
25+
npm registry (unauthenticated, `@objectstack/spec` + `@objectstack/cli` as
26+
positive controls, `@objectstack/service-ai-studio` — the sibling
27+
`edition: 'cloud'` entry — as a negative control returning 404):
28+
`@objectstack/service-ai` returns **200** with 57 versions, and the highest is
29+
**10.3.0** (2026-06-23) — entirely below the 11.3.0 cut the roster note names,
30+
i.e. the pre-cloud tail left behind on the registry. A determined operator
31+
following the old sentence installs a seven-major-old AI runtime that
32+
exact-pins `@objectstack/spec@10.3.0` against this repo's 17.2.0, resolving a
33+
second spec beside this one. The new message says so, so nobody discovers it
34+
from a dependency error.
35+
36+
The boundary sentence is **read from the roster**, not hand-written a fourth
37+
time: `note` is documented as "surfaced verbatim inside the preflight / boot
38+
error so the message carries its own context", and `packages/cli`'s capability
39+
preflight already interpolates it the same way. The three provider prefixes
40+
stay distinct — only the shared trailing sentence converges.
41+
42+
No behaviour change: `ok` and `severity` are untouched on every branch, and the
43+
embedder hint at the fourth site is deliberately left alone (its package
44+
**is** built here, so that instruction is followable as written) and pinned by
45+
a contrast test.

‎packages/services/service-settings/src/manifests/ai.manifest.test.ts‎

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22

33
import { describe, it, expect } from 'vitest';
44
import { SettingsManifestSchema } from '@objectstack/spec/system';
5+
import { PLATFORM_CAPABILITY_PROVIDERS } from '@objectstack/spec/kernel';
56
import { SettingsService } from '../settings-service.js';
67
import type { CryptoAdapter } from '../crypto-adapter.js';
78
import { aiSettingsManifest, aiTestActionHandler, aiTestEmbedderActionHandler } from './ai.manifest.js';
@@ -271,3 +272,77 @@ describe('aiSettingsManifest — embedder section', () => {
271272
});
272273
});
273274
});
275+
276+
/**
277+
* #11318 — the live-call hint an operator reads under "Test connection" in
278+
* Settings -> AI must carry the edition boundary the platform's own capability
279+
* roster already records. `PLATFORM_CAPABILITY_PROVIDERS.ai` declares
280+
* `edition: 'cloud'` ("no installable version in the open edition"), while the
281+
* hint used to say only "Mount @objectstack/service-ai to exercise live calls"
282+
* on all three real-provider branches.
283+
*
284+
* Pinned as a PROPERTY, not as a literal sentence: these assertions read the
285+
* roster entry here and require the message to carry its `package` and its
286+
* `note`, so the wording stays free to improve, but strip the note from spec —
287+
* or stop interpolating it — and they fail. A sentence copied into this file
288+
* would be one more hand-written copy of the very claim this card is about,
289+
* free to drift from spec with the test still green. Same shape PR #11266
290+
* established for the CLI's enterprise hint.
291+
*/
292+
describe('aiTestActionHandler — live-call hint carries the cloud boundary (#11318)', () => {
293+
const ai = PLATFORM_CAPABILITY_PROVIDERS.ai;
294+
295+
it('the roster still answers this question — the premise the rest rests on', () => {
296+
expect(ai, 'PLATFORM_CAPABILITY_PROVIDERS must still carry an `ai` entry').toBeTruthy();
297+
expect(ai.package).toBe('@objectstack/service-ai');
298+
expect(ai.edition).toBe('cloud');
299+
expect(ai.note, 'the cloud entry must carry an edition note to surface').toBeTruthy();
300+
});
301+
302+
// All three branches that print the hint, each exercised separately so a fix
303+
// applied to one cannot stand in for the others.
304+
const branches: Array<[string, Record<string, unknown>]> = [
305+
['the Vercel AI Gateway branch', { provider: 'gateway', gateway_model: 'openai/gpt-4o' }],
306+
[
307+
'the Cloudflare AI Gateway branch',
308+
{ provider: 'cloudflare', cloudflare_account_id: 'acct-1', cloudflare_api_key: 'cf-key' },
309+
],
310+
[
311+
'the plain SDK-provider branch',
312+
{ provider: 'openai', openai_api_key: 'sk-test', openai_model: 'gpt-4o' },
313+
],
314+
];
315+
316+
for (const [label, values] of branches) {
317+
it(`states the boundary on ${label}`, async () => {
318+
const r = await runTest(values);
319+
// Unchanged outcome: this is a boundary added to an advisory message,
320+
// not a new refusal.
321+
expect(r.ok).toBe(true);
322+
expect(r.severity).toBe('info');
323+
const message = String(r.message);
324+
// The instruction is KEPT — an operator may well have a cloud tier.
325+
expect(message).toContain(ai.package);
326+
// ...and now carries spec's own note, verbatim rather than re-typed.
327+
expect(message).toContain(ai.note!);
328+
expect(message).toMatch(/no open-edition version to install/);
329+
// The un-followable form this card retired: the package named, bare.
330+
expect(message).not.toMatch(/Mount @objectstack\/service-ai to exercise live calls/);
331+
});
332+
}
333+
334+
// Contrast control, so the assertions above cannot be satisfied by blanket-
335+
// rewording every hint in this file: the embedder hint names
336+
// `@objectstack/embedder-openai`, which IS built in this repo (8 path hits
337+
// under `git ls-tree -r --name-only HEAD | grep -cF /embedder-openai/`, against
338+
// 0 for `/service-ai/`), so that instruction is followable as written and stays
339+
// a plain mount line. Asserted here, deliberately not edited — #11318 fences
340+
// this site out by name.
341+
it('leaves the embedder hint a plain mount line — its package IS built here', async () => {
342+
const r = await runTestEmbedder({ embedder_provider: 'openai', embedder_api_key: 'sk-test' });
343+
expect(r.ok).toBe(true);
344+
expect(String(r.message)).toContain(
345+
'Mount @objectstack/embedder-openai + a knowledge adapter to exercise live calls',
346+
);
347+
});
348+
});

‎packages/services/service-settings/src/manifests/ai.manifest.ts‎

Lines changed: 52 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license.
22

33
import type { SettingsManifest } from '@objectstack/spec/system';
4+
import { PLATFORM_CAPABILITY_PROVIDERS } from '@objectstack/spec/kernel';
45
import type { SettingsActionHandler } from '../settings-service.types.js';
56

67
// Visibility expressions are written as inline strings here for
@@ -307,6 +308,54 @@ const manifest = {
307308
/** AI — provider / model / credentials configuration. */
308309
export const aiSettingsManifest = manifest as unknown as SettingsManifest;
309310

311+
/**
312+
* The live-call hint the three real-provider branches of `ai/test` end with.
313+
*
314+
* An operator who configures a provider here and presses "Test connection" was
315+
* told, in three places, to "Mount @objectstack/service-ai to exercise live
316+
* calls" — an instruction this platform's own capability roster contradicts.
317+
* `PLATFORM_CAPABILITY_PROVIDERS.ai` declares `edition: 'cloud'`, which
318+
* `CapabilityEdition` defines as "realized only by a cloud runtime tier; there
319+
* is **no installable version in the open edition**": exactly the un-followable
320+
* "add it to your dependencies" that framework#3366 exists to make legible. The
321+
* instruction is kept — an operator may well have a cloud tier — and gains the
322+
* boundary it was missing, so one who does not can see the path is closed.
323+
*
324+
* Read FROM the roster rather than hand-written a fourth time. The `note` field
325+
* is documented as "surfaced verbatim inside the preflight / boot error so the
326+
* message carries its own context", and `packages/cli`'s capability preflight
327+
* already interpolates it this way (#10921 / PR #11266); a copy here could drift
328+
* from spec with both still green. Deliberately no tolerant fallback for a
329+
* missing entry: a hand-written substitute silently replacing the spec-owned
330+
* claim is the very defect this card is about, and three tests across two
331+
* packages pin the entry's presence.
332+
*
333+
* The registry half is what the roster does NOT say, and it is why the hint
334+
* cannot stop at "requires cloud". Measured 2026-08-23, unauthenticated public
335+
* npm registry, with `@objectstack/spec` + `@objectstack/cli` as positive
336+
* controls and `@objectstack/service-ai-studio` — the sibling `edition: 'cloud'`
337+
* entry — as a negative control (404, so a 200 here is not "cloud packages are
338+
* published anyway"): `@objectstack/service-ai` returns **200**, 57 versions,
339+
* highest **10.3.0** (2026-06-23) — entirely below the 11.3.0 cut the note
340+
* names, i.e. the pre-cloud tail left on the registry. So the install does not
341+
* 404; it SUCCEEDS, pinning a seven-major-old runtime, and 10.3.0 exact-pins
342+
* `@objectstack/spec@10.3.0` against this repo's 17.2.0, resolving a second spec
343+
* beside this one. Succeeding wrongly is worse than failing, so the hint says
344+
* so. The sentence carries none of those numbers — it derives its boundary from
345+
* the roster note — but re-measure before treating them as current.
346+
*/
347+
const AI_RUNTIME_HINT = (() => {
348+
const ai = PLATFORM_CAPABILITY_PROVIDERS.ai;
349+
const note = ai.note ? ` (${ai.note})` : '';
350+
return (
351+
`Live calls need ${ai.package}${note}, which this platform does not ship: it is ` +
352+
'realized only by an ObjectStack Cloud tier, so there is no open-edition version to ' +
353+
'install. The name does still resolve on the public npm registry, but only at ' +
354+
'versions from before that cut — installing it pins a pre-cloud AI runtime and a ' +
355+
'second @objectstack/spec beside this one instead of enabling AI here.'
356+
);
357+
})();
358+
310359
/**
311360
* Built-in fallback action handler for `ai/test`. The real
312361
* implementation that issues a live `chat()` round-trip lives in
@@ -340,7 +389,7 @@ export const aiTestActionHandler: SettingsActionHandler = async ({ values, paylo
340389
return {
341390
ok: true,
342391
severity: 'info',
343-
message: `Vercel AI Gateway configured (model=${values.gateway_model}). Mount @objectstack/service-ai to exercise live calls.`,
392+
message: `Vercel AI Gateway configured (model=${values.gateway_model}). ${AI_RUNTIME_HINT}`,
344393
};
345394
}
346395
// Cloudflare needs more than just an API key.
@@ -355,7 +404,7 @@ export const aiTestActionHandler: SettingsActionHandler = async ({ values, paylo
355404
return {
356405
ok: true,
357406
severity: 'info',
358-
message: `Cloudflare AI Gateway configured (model=${model}). Mount @objectstack/service-ai to exercise live calls.`,
407+
message: `Cloudflare AI Gateway configured (model=${model}). ${AI_RUNTIME_HINT}`,
359408
};
360409
}
361410
const keyField = `${provider}_api_key`;
@@ -367,7 +416,7 @@ export const aiTestActionHandler: SettingsActionHandler = async ({ values, paylo
367416
return {
368417
ok: true,
369418
severity: 'info',
370-
message: `${provider} configured (model=${model}). Mount @objectstack/service-ai to exercise live calls.`,
419+
message: `${provider} configured (model=${model}). ${AI_RUNTIME_HINT}`,
371420
};
372421
};
373422

0 commit comments

Comments
 (0)