Umbrella: OpenManus research adoption — browser perception loop, budgets, external MCP #11536

Context

/research https://github.com/FoundationAgents/OpenManus (2026-07-10) compared OpenManus (FoundationAgents, ~57k stars, MetaGPT team) against AutoBot. Most of OpenManus is weaker than what we have (sandboxing, planning, LLM resilience, fetch/crawl pipeline), but the comparison surfaced 8 concrete adoptables/gaps, each verified against our code with file:line evidence.

The unifying theme: OpenManus runs a closed perception-action loop for browsing (indexed clickable elements + screenshot injected into every LLM step), while AutoBot's LLM browses blind — guessing CSS selectors against pages it can't see, with screenshots reduced to the text "Screenshot captured successfully" (autobot-backend/chat_workflow/tool_handler.py:2042-2046).

Tasks

• feat(browser): indexed interactive-element interaction — click/fill by index from accessibility snapshot #11537 (HIGH): Indexed interactive-element browsing — click/fill by index from accessibility snapshot
• feat(chat): vision-in-the-loop — thread browser/VNC screenshots into LLM chat context #11538 (HIGH): Vision-in-the-loop — thread browser/VNC screenshots into LLM chat context
• fix(browser-worker): per-conversation browser session isolation — cookies/login state bleed across conversations via shared navPage #11539 (HIGH): Per-conversation browser session isolation — fix session bleed via shared navPage
• feat(chat): live-page extract_content builtin — goal-directed extraction on the current browser session #11540 (MEDIUM): Live-page extract_content builtin (goal-directed extraction on the live browser session)
• feat(llm): pre-request cumulative token budget gate at the provider seam #11541 (MEDIUM): Pre-request cumulative token budget gate at the LLM seam
• feat(mcp): generic external MCP server client bridge — user-configured stdio/SSE servers #11542 (MEDIUM): Generic external MCP server client bridge (stdio/SSE)
• harden(chat): terminal hard-size cap for unmatched tool output in ToolOutputFilter #11543 (LOW, quick-win): Hard fallback size cap for unmatched tool output in ToolOutputFilter
• feat(browser): ergonomic actions — scroll_to_text, get_dropdown_options, multi-tab support #11544 (LOW): Browser ergonomics — scroll_to_text, get_dropdown_options, multi-tab (depends on fix(browser-worker): per-conversation browser session isolation — cookies/login state bleed across conversations via shared navPage #11539)

Suggested order

#11539 first (security-adjacent session bleed, and #11544 tabs depend on its per-session contexts), then #11537 + #11538 together (they form the perception loop), then the rest independently.

Source references

• OpenManus perception loop: app/agent/browser.py:47-79, app/tool/browser_use_tool.py:479-537
• OpenManus token budget: app/llm.py:240-262 (cumulative pre-request check, non-retryable TokenLimitExceeded)
• OpenManus MCP client: app/agent/manus.py:67-129 (SSE/stdio, hot-merge per server ID)