Skip to content

feat(runtime-host): execute hosted sessions with real models - #1637

Merged
Astro-Han merged 3 commits into
apache:mainfrom
M4n5ter:feat/runtime-host-real-model-execution
Jul 30, 2026
Merged

Astro-Han merged 3 commits into
apache:mainfrom
M4n5ter:feat/runtime-host-real-model-execution

Conversation

@M4n5ter

@M4n5ter M4n5ter commented Jul 30, 2026 •

Copy link
Copy Markdown
Member
English

Summary

Enable Runtime Host to execute hosted Sessions against real providers through AI SDK backends constructed from canonical Runtime Policy state.

Execution authority

  • Resolve an enabled, chat-capable model from the canonical connection catalog.
  • Read credentials and proxy configuration from their authoritative stores.
  • Support canonical API-key, optional-key, and no-auth provider configurations.
  • Fail closed for OAuth-backed execution until Runtime Host owns that authority.
  • Create immutable direct or proxied transports per backend and close them with the backend lifecycle.

Prompt and tools

  • Build each root prompt from one coherent Runtime Policy and Memory snapshot.
  • Include personalization, workspace instructions, the canonical Skill inventory, and the Task Ledger.
  • Share one turn-scoped Skill inventory across the system prompt, Skill, and SkillSearch; refresh it on the next turn.
  • Exclude root personalization and Memory from child prompts.
  • Expose only Runtime Host-supported pure tools and retain deferred tool loading.

Attribution and lifecycle

  • Persist provider request capture artifacts and attempt records before every physical main or history-compaction request.
  • Attribute usage and pricing separately through main and history_compact call kinds.
  • Register backend invalidation before committed Runtime Policy, Connection, or Pricing mutations release their activation fence.
  • Allow an active turn to finish while ensuring the next backend activation waits for invalidation completion.
  • Poison and drain the Host when request capture, authoritative persistence, or backend invalidation fails.

Boundaries

This change does not add OAuth execution, Runtime Resource, Automation, Goal, Native, or Browser authority. It also does not switch Desktop or CLI production routing to Runtime Host.

Validation

  • Core: 1,193 passed
  • Storage: 806 passed, 1 skipped
  • Runtime: 2,799 passed, 9 skipped
  • Runtime Host: 360 passed
  • CLI: 672 passed
  • Backend refresh regression tests: 6 passed
  • Runtime Policy integration tests: 12 passed
  • Root typecheck and production build
  • Biome lint and format checks
  • Real local OpenAI-compatible HTTP/SSE provider integration
  • Real OpenRouter openrouter/free execution through the production Runtime Host composition
  • git diff --check

Part of #1167 and #853.

简体中文

概要

让 Runtime Host 能够根据权威 Runtime Policy 状态构造 AI SDK backend,并通过真实 provider 执行托管 Session。

执行权威

  • 从权威 Connection Catalog 中解析已启用且支持对话的模型。
  • 从各自的权威 Store 读取凭据和代理配置。
  • 支持权威配置中的 API Key、可选 Key 和无认证 provider。
  • 在 Runtime Host 尚未拥有 OAuth 执行权威时保持失败关闭。
  • 为每个 backend 创建不可变的直连或代理 transport,并随 backend 生命周期关闭。

Prompt 与工具

  • 使用同一份一致的 Runtime Policy 和 Memory 快照构造每个 root prompt。
  • 包含个性化设置、workspace instructions、权威 Skill inventory 和 Task Ledger。
  • system prompt、Skill 和 SkillSearch 在同一 turn 内共享一份 Skill inventory,并在下一 turn 刷新。
  • child prompt 不继承 root personalization 和 Memory。
  • 仅暴露 Runtime Host 已支持的纯工具,并保留延迟加载机制。

归因与生命周期

  • 在每次真实的 main 或 history-compaction provider 请求前持久化 request capture artifact 和 attempt record。
  • 使用 main 和 history_compact 分别记录 usage 与 pricing。
  • Runtime Policy、Connection 或 Pricing mutation 提交后,在释放 activation fence 前登记 backend invalidation。
  • 允许当前 active turn 正常结束,同时确保下一次 backend activation 等待 invalidation 完成。
  • request capture、权威持久化或 backend invalidation 失败时,Runtime Host 会进入 poison 和 drain 状态。

边界

本次变更不增加 OAuth 执行、Runtime Resource、Automation、Goal、Native 或 Browser 权威,也不切换 Desktop 或 CLI 的生产执行路由。

验证

  • Core:1,193 项通过
  • Storage:806 项通过,1 项跳过
  • Runtime:2,799 项通过,9 项跳过
  • Runtime Host:360 项通过
  • CLI:672 项通过
  • Backend refresh 回归测试:6 项通过
  • Runtime Policy 集成测试:12 项通过
  • 根级 typecheck 与生产构建
  • Biome lint 与 format 检查
  • 本地真实 OpenAI-compatible HTTP/SSE provider 集成
  • 通过生产 Runtime Host composition 执行真实 OpenRouter openrouter/free 调用
  • git diff --check

属于 #1167 和 #853 的一部分。

@M4n5ter
M4n5ter marked this pull request as ready for review July 30, 2026 02:22
@Astro-Han

Copy link
Copy Markdown
Contributor

I found two lifecycle issues:

  • P1: createHostAiSdkBackend does not observe BackendFactoryContext.abortSignal while waiting for connection resolution or the pricing snapshot. If either read stalls, interrupting a turn that is still starting never settles. RuntimeKernel has no outer cancellation race; it aborts the signal and waits for the factory claim. Please make both reads abortable and cover stalled connection and pricing reads.
  • P2: beginDrain() closes usage admission before active turns stop. Provider and tool usage is written as those turns settle, so usage already incurred can be rejected and silently disappear from the canonical log. Keep admission open until active turns and backends settle, then drain the usage store.

@M4n5ter

M4n5ter commented Jul 30, 2026

Copy link
Copy Markdown
Member Author

@Astro-Han Both lifecycle issues above have been fixed in 696a5e9: backend creation now aborts stalled connection/pricing reads, and usage admission remains open until active turns and backends settle. Regression coverage was added for both cases, and CI is green. Please take another look when convenient.

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rechecked 696a5e915. Both earlier issues are fixed. Backend creation now observes cancellation while connection and pricing reads are pending. Usage admission also stays open until active turns and backends settle. The new tests cover both paths. No findings.

@Astro-Han
Astro-Han merged commit 7a5aea2 into apache:main Jul 30, 2026
3 checks passed
@M4n5ter
M4n5ter deleted the feat/runtime-host-real-model-execution branch July 30, 2026 06:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants