Send insights with the engine's own key, and make /v1/chat/completions work

[flxkm522]

What this changes

Adds POST /api/insights/sessions. A product hands the engine a batch of labeled AI work sessions, and the engine posts it to the Redrob Console's insights using the engine's own Redrob key. The product gets back the Console's { accepted, updated, rejected }.

It's for Redrob Office, whose AGENTS.md rule is "Office never holds a provider key". This keeps that rule: the key never leaves the engine.

• Fixed destination. The batch always goes to ${CONSOLE_URL}/insights/sessions, the same constant every Console call here uses, including its developer override. The route takes no URL, and an extra url field in the body is ignored.
• The key never appears in a reply, a log or an error message. It's resolved with ConsoleKey.resolve() (environment, then the credential store, then auth.json), as the variants and chat-completions routes do.
• Errors, mapped like the variants routes:
  • 404 ProviderNotFoundError when the engine has no Redrob key;
  • 401 for a refused or over-budget key;
  • 400 for a refused batch, carrying the Console's message (NestJS's list form included);
  • 503 for 429, 5xx or an unreachable Console.
• No retries. The product keeps a batch that got no answer for its next sync.
• Sessions are passed through as objects. The Console owns their shape and refuses any field that isn't a label, so a copy of the schema here would only drift. This side checks only the batch size: 1 to 500 sessions, the Console's own limit.

It follows the variants route's layout: schema/src/insights.ts, core/src/insights.ts, protocol/src/groups/insights.ts, server/src/handlers/insights.ts, plus registration, the regenerated client, and a section in docs/LOCAL-ENGINE-API.md.

Also in this PR: /v1/chat/completions works, and carries the session id

Office's next step is to send its chat through the engine instead of calling Console with a key it holds. On develop that route does not work:

• Every request answered 500 Service not found: @redrob/LLMClient. Neither server assembly provided the LLM client. It now does, next to httpClient (packages/redrob/.../httpapi/server.ts and packages/server/src/routes.ts).
• Every streamed reply was an empty 200 (Content-Length: 0). LLM.stream looks the client up when the body starts, after the handler has returned. The handler now takes the client itself and calls client.stream.
• The exerciser only covers the route's 400 path (no provider credential in CI), which is why neither showed.

It also forwards x-redrob-session, so Console can join a labeled session to what its requests cost. Only for the Redrob provider, and only in Console's own id format (1–128 of A-Za-z0-9._:-), so it is an id, never text, and never goes to another vendor.

Why

Office needs to send the labels its sessions get on the person's machine to the Console's insights, with the person's key. The design discussion is in redrob-cowork#149.

• Reading the key back over GET /provider would mean Office holds it in memory every time it sends. That's what Office's EngineIntegrationClient was built to rule out: "There is no read path for the key itself. A caller that cannot read a key cannot log it, sync it, or leak it."
• With this route, the engine makes the call: the same shape as "the engine makes the call; Office names a model and gets an answer".

How it was verified

• bun test test/insights.test.ts (packages/core): exit 0, 4 pass. A stand-in Console records what it receives:

  • the stored key goes in Authorization: Bearer … to https://console.redrob.ai/api/backend/v1/insights/sessions, with the batch as sent, and appears nowhere in the result;
  • per-session rejections pass through;
  • with no key, the route fails as not connected and the Console is never called;
  • a 400's message (NestJS list form) is carried through.
  • Mutation check: with the Bearer header removed, a test fails (3 pass, 1 fail).

• The real engine, end to end. bun run ./src/index.ts serve, with a fake Console behind REDROB_CONSOLE_URL, driven with curl:

  Case	Answer
  No key	404 ProviderNotFoundError
  No Basic auth	401
  Batch of 2	200 {"accepted":2,"updated":0,"rejected":[]}
  Session with a summary field	400 "sessions.0.summary is not a field" (the Console's message)
  Empty batch	400
  Body with an extra "url": "https://evil.test/collect"	the batch still went to the Console only

  The Console received Bearer rrk_e2e_secret with only sessions in each body. The key appeared in 0 replies and 0 engine log lines.

• test/server/chat-completion-session-header.test.ts (packages/redrob): 3 pass. Forwarded in the id format to Redrob; dropped when it is text, too long or absent; never sent to openai/… or ollama/….

• Real engine, chat end to end, with a fake Console answering SSE behind REDROB_CONSOLE_URL and REDROB_API_KEY set:

  • before the fix: 500 Service not found: @redrob/LLMClient, and the Console was never called;
  • after: non-streamed 200 with the completion; streamed 200 with content: "hi", finish_reason: "stop", then [DONE];
  • the Console received x-redrob-session: of_0123abcd and of_stream1 where they were sent, nothing for an invalid or absent one, and Bearer rrk_chat each time.

• bun run test:httpapi (packages/redrob): after the chat fix, all three modes exit 0 (216 pass each). Earlier runs: the new route passes in all three modes, through its not-connected path, as the variants are exercised. The command still exits 1, because session.shell times out in --mode effect. That also happens on develop (baseline --mode effect: exit 1, the same session.shell failure). Coverage and auth modes: 216 pass, 0 fail, 0 missing.

• bun typecheck (root, turbo): exit 0, 18/18 tasks.

• bun run check:generated (packages/client): exit 0 after committing the regenerated client.

• bun test (packages/core): 1037 pass, 3 fail. The 3 also fail on develop here, because this sandbox runs as root (util.flock and util.effect-flock "unwritable lock roots", and ProjectV2.resolve ssh/https remotes).

• bun install --frozen-lockfile with the pinned bun 1.3.14: exit 0. The pre-push hook ran.

Anything a reviewer should be wary of

• Not regenerated: the JS SDK (packages/sdk/js/src/v2/gen). Regenerating it adds about 650 lines, most of them routes that are missing there already (the browser-request routes, for example). CI doesn't check it, and mixing that catch-up into this PR would hide the change. It's worth doing as its own change.
• Reaching users: this needs an engine release and a pin bump in each product that bundles the engine (root AGENTS.md).
• Office's side, which calls this route, comes next.

[flxkm522]

Added chat.completions.reply and chat.completions.stream to the httpapi exerciser. They resolve the fake test provider (test/test-model, the same TestLLMServer the session scenarios use) and assert the answer, both streamed and not. Until now only the 400 path was exercised, which is how both bugs reached develop.

They catch each bug. With llmClient removed from the server, both fail. With LLM.stream restored in the handler, the stream scenario fails.

bun run test:httpapi: exit 0, 218 pass in each of the three modes.