fix(backend/blocks): orchestrator EXTENDED_THINKING emits final-answer only

[majdyz]

Why / What / How

Why: Discovered while debugging a debate-style orchestrator agent on dev preview. The two execution modes were producing structurally different finished outputs for the same block:

Mode	finished content	What's wrong with it
BUILT_IN (agent loop via tool_call_loop)	response.response_text from the last LLM call (i.e. the model's final text-only response after it stopped calling tools)	Correct — this is the agent's composed answer
EXTENDED_THINKING (Claude Agent SDK)	"".join(every TextBlock from every AssistantMessage)	Buggy — accumulates every narration emitted between tool calls; the answer gets buried in working-log noise, and worse, masks prompts that never compose a real answer

The original-direction commit (0f4d348d0d, now force-replaced) tried to fix the divergence by making BUILT_IN match EXTENDED_THINKING's accumulation. That was the wrong direction: BUILT_IN's last-answer behaviour is the correct contract, and the accumulation in EXTENDED_THINKING was hiding composition bugs that dry-run / autopilot should be able to detect.

What: Replace response_parts.append(...) (every TextBlock) with final_response_parts = list(text_parts) only when the assistant message has no tool calls — i.e. the model has stopped calling tools and is emitting its composed answer. The last such message wins.

How:

- response_parts: list[str] = []
+ final_response_parts: list[str] = []   # only the final-answer message's text
  ...
  if isinstance(sdk_msg, AssistantMessage):
      text_parts = []
      tool_use_parts = []
      for content_block in sdk_msg.content:
          if isinstance(content_block, TextBlock):
              text_parts.append(content_block.text)
-             response_parts.append(content_block.text)
          elif isinstance(content_block, ToolUseBlock):
              ...
+     # Capture the agent's final answer: last text-only message wins.
+     if text_parts and not tool_use_parts:
+         final_response_parts = list(text_parts)
  ...
- response_text = "".join(response_parts)
+ response_text = "".join(final_response_parts)

Edge cases:

• Agent never stops calling tools (e.g. hits max iterations): final_response_parts stays empty → finished = "". This is the correct signal for dry-run / autopilot diagnostics: the prompt didn't compose a final answer, so the agent author needs to fix it. Previously the over-accumulation returned a transcript-shaped string that masked this failure mode.
• Agent emits multiple text-only messages over the course of the run (rare): the last one wins, matching "the model's most recent composed answer."
• Agent emits text + tool calls in the same message: that message's text is treated as narration belonging to conversations, not the answer (the not tool_use_parts guard).

Existing tests: all 14 in test_orchestrator_execution_mode.py still pass. No new SDK-mocking infrastructure exists in the test suite for this code path, so behavioural verification will happen on the next dev preview redeploy (combo branch redeployment to follow).

Changes 🏗️

• _execute_tools_sdk_mode captures only the last assistant message's text-only content into final_response_parts.
• finished output yields "".join(final_response_parts) — empty string when the agent never composed a final answer (useful diagnostic signal).
• No change to conversations output (still carries the full back-and-forth).

Checklist 📋

For code changes:

• I have clearly listed my changes in the PR description
• I have made a test plan
• I have tested my changes according to the test plan:
  • poetry run pytest backend/blocks/test/test_orchestrator_execution_mode.py — 14 tests pass
  • poetry run black + poetry run ruff check clean
  • Manual verification on dev preview after combo redeploy

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 50813f7a-a616-4bdd-ba9d-8b2c547d8177

📥 Commits

Reviewing files that changed from the base of the PR and between a67da57 and 1e07f32.

📒 Files selected for processing (2)

• autogpt_platform/backend/backend/blocks/orchestrator.py
• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py

📜 Recent review details

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)

• GitHub Check: check API types
• GitHub Check: end-to-end tests
• GitHub Check: test (3.12)
• GitHub Check: test (3.13)
• GitHub Check: test (3.11)
• GitHub Check: Seer Code Review
• GitHub Check: Check PR Status

🧰 Additional context used

📓 Path-based instructions (5)

autogpt_platform/backend/**/*.py

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

autogpt_platform/backend/**/*.py: Use Python 3.11 (required; managed by Poetry via pyproject.toml) for backend development
Always run 'poetry run format' (Black + isort) before linting in backend development
Always run 'poetry run lint' (ruff) after formatting in backend development

autogpt_platform/backend/**/*.py: Use poetry run ... command for executing Python package dependencies
Use top-level imports only — avoid local/inner imports except for lazy imports of heavy optional dependencies like openpyxl
Use absolute imports with from backend.module import ... for cross-package imports; single-dot relative imports are acceptable for sibling modules within the same package; avoid double-dot relative imports
Do not use duck typing — avoid hasattr/getattr/isinstance for type dispatch; use typed interfaces/unions/protocols instead
Use Pydantic models over dataclass/namedtuple/dict for structured data
Do not use linter suppressors — no # type: ignore, # noqa, # pyright: ignore; fix the type/code instead
Prefer list comprehensions over manual loop-and-append patterns
Use early return with guard clauses first to avoid deep nesting
Use %s for deferred interpolation in debug log statements for efficiency; use f-strings elsewhere for readability (e.g., logger.debug("Processing %s items", count) vs logger.info(f"Processing {count} items"))
Sanitize error paths by using os.path.basename() in error messages to avoid leaking directory structure
Be aware of TOCTOU (Time-Of-Check-Time-Of-Use) issues — avoid check-then-act patterns for file access and credit charging
Use transaction=True for Redis pipelines to ensure atomicity on multi-step operations
Use max(0, value) guards for computed values that should never be negative
Keep files under ~300 lines; if a file grows beyond this, split by responsibility (extract helpers, models, or a sub-module into a new file)
Keep functions under ~40 lines; extract named helpers when a function grows longer
...

Files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

autogpt_platform/backend/backend/blocks/**/*.py

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

autogpt_platform/backend/backend/blocks/**/*.py: Inherit from 'Block' base class with input/output schemas when adding new blocks in backend
Implement 'run' method with proper error handling in backend blocks
Generate block UUID using 'uuid.uuid4()' when creating new blocks in backend
Write tests alongside block implementation when adding new blocks in backend

autogpt_platform/backend/backend/blocks/**/*.py: For blocks handling files, use store_media_file() with return_format="for_local_processing" when processing with local tools (ffmpeg, MoviePy, PIL)
For blocks handling files, use store_media_file() with return_format="for_external_api" when sending content to external APIs (Replicate, OpenAI)
For blocks returning files, use store_media_file() with return_format="for_block_output" to enable auto-adaptation to execution context (workspace:// in CoPilot, data URI in graphs)
When creating new blocks, inherit from Block base class, define input/output schemas using BlockSchema, implement async run method, and generate unique block ID using uuid.uuid4()

Files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

autogpt_platform/backend/**/test/**/*.py

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

Use snapshot testing with '--snapshot-update' flag in backend tests when output changes; always review with 'git diff'

Files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py

autogpt_platform/{backend,autogpt_libs}/**/*.py

📄 CodeRabbit inference engine (AGENTS.md)

Format Python code with poetry run format

Files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

autogpt_platform/backend/**/test_*.py

📄 CodeRabbit inference engine (autogpt_platform/AGENTS.md)

Create a failing test first using @pytest.mark.xfail decorator (backend) when fixing a bug or adding a feature, then implement the fix and remove the xfail marker

Files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py

🧠 Learnings (15)

📚 Learning: 2026-02-05T04:11:00.596Z

Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 11796
File: autogpt_platform/backend/backend/blocks/video/concat.py:3-4
Timestamp: 2026-02-05T04:11:00.596Z
Learning: In autogpt_platform/backend/backend/blocks/**/*.py, when creating a new block, generate a UUID once with uuid.uuid4() and hard-code the resulting string as the block's id parameter. Do not call uuid.uuid4() at runtime; IDs must be constant across all imports and runs to ensure stability.

Applied to files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

📚 Learning: 2026-03-16T16:32:21.686Z

Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/pods.py:62-74
Timestamp: 2026-03-16T16:32:21.686Z
Learning: In autogpt_platform/backend/backend/blocks/, the Block base class execute() already wraps run() in a try/except to convert uncaught exceptions into BlockExecutionError/BlockUnknownError. Do not add per-block try/except in individual block run() methods, as this is not the established pattern (e.g., Gmail, Slack, Todoist blocks omit it). Only use explicit try/except within blocks that need to distinguish between success and error yield paths inside a generator (e.g., attachment blocks). This guidance applies to all Python files under autogpt_platform/backend/backend/blocks/ and similar block implementations; avoid duplicating error handling in run() unless a block requires generator-based branching.

Applied to files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

📚 Learning: 2026-04-23T12:55:26.122Z

Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12893
File: autogpt_platform/backend/backend/blocks/ayrshare/post_to_tiktok.py:24-24
Timestamp: 2026-04-23T12:55:26.122Z
Learning: Cost billing via the cost(*costs) decorator is applied at input-evaluation time (before a block’s run() executes). Therefore, mutating input_data inside run() will not change billing. When a block’s billing depends on a field plus URL/sniff-derived signals, treat the explicitly declared billing field (e.g., is_video) as the only billing source—set it correctly before run() (or in the code path that occurs before the decorator evaluates input_data). This should be checked for all blocks under autogpt_platform/backend/backend/blocks/ so billing signals are not mistakenly assumed to update during run().

Applied to files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

📚 Learning: 2026-02-26T17:02:22.448Z

Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12211
File: .pre-commit-config.yaml:160-179
Timestamp: 2026-02-26T17:02:22.448Z
Learning: Keep the pre-commit hook pattern broad for autogpt_platform/backend to ensure OpenAPI schema changes are captured. Do not narrow to backend/api/ alone, since the generated schema depends on Pydantic models across multiple directories (backend/data/, backend/blocks/, backend/copilot/, backend/integrations/, backend/util/). Narrowing could miss schema changes and cause frontend type desynchronization.

Applied to files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

📚 Learning: 2026-03-05T15:42:08.207Z

Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12297
File: .claude/skills/backend-check/SKILL.md:14-16
Timestamp: 2026-03-05T15:42:08.207Z
Learning: In Python files under autogpt_platform/backend (recursively), rely on poetry run format to perform formatting (Black + isort) and linting (ruff). Do not run poetry run lint as a separate step after poetry run format, since format already includes linting checks.

Applied to files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

📚 Learning: 2026-03-16T16:30:11.452Z

Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/threads.py:80-102
Timestamp: 2026-03-16T16:30:11.452Z
Learning: In autogpt_platform/backend/backend/blocks/ (and related blocks under autogpt_platform/backend/backend/blocks/), do not add try/except blocks around a block's run() method for standard error propagation. The block executor framework (backend/executor/manager.py) catches uncaught exceptions from run() and emits them on the 'error' output. Only add explicit try/except blocks when you need to control partial outputs in failure cases (e.g., certain outputs must not be yielded on error, as in attachment blocks). This is the standard pattern across the codebase; apply it broadly to blocks' run() implementations.

Applied to files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

📚 Learning: 2026-03-16T16:30:23.196Z

Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/pods.py:62-74
Timestamp: 2026-03-16T16:30:23.196Z
Learning: In any Python file under autogpt_platform/backend/backend/blocks, do not add a try/except around run() solely for standard error handling. The block framework’s _execute() in _base.py already catches unhandled exceptions and re-raises as BlockExecutionError or BlockUnknownError. If you yield ("error", message), _execute() raises BlockExecutionError immediately, so the error port will not propagate downstream. Reserve explicit try/except for scenarios where you must control partial output (e.g., attachment blocks that must skip yielding content_base64 on failure).

Applied to files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

📚 Learning: 2026-03-16T16:30:11.452Z

Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/threads.py:80-102
Timestamp: 2026-03-16T16:30:11.452Z
Learning: Do not wrap synchronous AgentMail SDK calls with asyncio.to_thread() in blocks under autogpt_platform/backend/backend/blocks (and across the codebase). The block executor runs node execution in dedicated threads via asyncio.run_coroutine_threadsafe (see manager.py around lines ~745-752 and ~1079). The existing pattern avoids using asyncio.to_thread for SDK calls inside async run() methods, so maintain that approach and do not add to_thread usage in these code paths.

Applied to files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

📚 Learning: 2026-03-16T16:35:40.236Z

Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/api/features/workflow_import.py:54-63
Timestamp: 2026-03-16T16:35:40.236Z
Learning: Avoid using the word 'competitor' in public-facing identifiers and text. Use neutral naming for API paths, model names, function names, and UI text. Examples: rename 'CompetitorFormat' to 'SourcePlatform', 'convert_competitor_workflow' to 'convert_workflow', '/competitor-workflow' to '/workflow'. Apply this guideline to files under autogpt_platform/backend and autogpt_platform/frontend.

Applied to files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

📚 Learning: 2026-03-19T15:10:50.676Z

Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12483
File: autogpt_platform/backend/backend/copilot/tools/test_dry_run.py:298-303
Timestamp: 2026-03-19T15:10:50.676Z
Learning: When using Python’s `unittest.mock.patch` in tests, choose the patch target based on how the imported name is resolved:
- If the code under test uses an **eager/module-level import** (e.g., `from foo.bar import baz` at module top), patch **the module where the name is looked up** (i.e., where it is used in the SUT), e.g. `patch("mymodule.baz")`.
- If the code under test uses a **lazy import** executed later (e.g., `from foo.bar import baz` inside a function/branch), patch **the source module** (e.g., `patch("foo.bar.baz")`) because the late `from ... import` will read the (potentially patched) name from the source module at call time.

For a concrete example: if `simulate_block` is imported inside an `if dry_run:` block in the SUT, then the correct test patch target is the source module path for `simulate_block` as it exists at call time (e.g., `patch("backend.executor.simulator.simulate_block")`), not the test file’s import location.

Applied to files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py

📚 Learning: 2026-03-31T15:37:38.626Z

Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12623
File: autogpt_platform/backend/backend/copilot/tools/agent_generator/fixer.py:37-47
Timestamp: 2026-03-31T15:37:38.626Z
Learning: When validating/constructing Anthropic API model IDs in Significant-Gravitas/AutoGPT, allow the hyphen-separated Claude Opus 4.6 model ID `claude-opus-4-6` (it corresponds to `LlmModel.CLAUDE_4_6_OPUS` in `autogpt_platform/backend/backend/blocks/llm.py`). Do NOT require the dot-separated form in Anthropic contexts. Only OpenRouter routing variants should use the dot separator (e.g., `anthropic/claude-opus-4.6`); `claude-opus-4-6` should be treated as correct when passed to Anthropic, and flagged only if it’s used in the OpenRouter path where the dot form is expected.

Applied to files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

📚 Learning: 2026-04-15T02:43:36.890Z

Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12780
File: autogpt_platform/backend/backend/copilot/tools/workspace_files.py:0-0
Timestamp: 2026-04-15T02:43:36.890Z
Learning: When reviewing Python exception handlers, do not flag `isinstance(e, X)` checks as dead/unreachable if the caught exception `X` is a subclass of the exception type being handled. For example, if `X` (e.g., `VirusScanError`) inherits from `ValueError` (directly or via an intermediate class) and it can be raised within an `except ValueError:` block, then `isinstance(e, X)` inside that handler is reachable and should not be treated as dead code.

Applied to files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

📚 Learning: 2026-04-22T11:46:04.431Z

Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12881
File: autogpt_platform/backend/backend/copilot/config.py:0-0
Timestamp: 2026-04-22T11:46:04.431Z
Learning: Do not flag the Claude Sonnet 4.6 model ID as incorrect when it uses the project’s established hyphenated convention: `anthropic/claude-sonnet-4-6`. This hyphen form is the intentional, production convention and should be treated as valid (including in files like llm.py, blocks tests, reasoning.py, `_is_anthropic_model` tests, and config defaults). Note that OpenRouter also accepts the dot variant `anthropic/claude-sonnet-4.6`, so either form may be tolerated, but `anthropic/claude-sonnet-4-6` should be considered the standard to match project usage.

Applied to files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

📚 Learning: 2026-04-22T11:46:12.892Z

Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12881
File: autogpt_platform/backend/backend/copilot/baseline/service.py:322-332
Timestamp: 2026-04-22T11:46:12.892Z
Learning: In this codebase (Significant-Gravitas/AutoGPT), OpenRouter-routed Anthropic model IDs should use the hyphen-separated convention (e.g., `anthropic/claude-sonnet-4-6`, `anthropic/claude-opus-4-6`). Although OpenRouter may accept both hyphen and dot variants, treat the hyphen-separated form as the intended, correct codebase-wide convention and do not flag it as an error. Only flag the dot-separated variant (e.g., `anthropic/claude-sonnet-4.6`) as incorrect when reviewing/validating model ID strings for OpenRouter-routed Anthropic models.

Applied to files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

📚 Learning: 2026-05-07T18:48:14.242Z

Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 13040
File: autogpt_platform/backend/backend/blocks/llm.py:0-0
Timestamp: 2026-05-07T18:48:14.242Z
Learning: In this repository, isort may split imports from the same module into separate blocks when some imports are aliased (e.g., `from module import X as Y`) and others are not. Preserve the two-block layout when it results from isort (such as keeping `from openai.types.chat import ChatCompletion as OpenAIChatCompletion` separate from non-aliased imports from `openai.types.chat`). Do not treat that split as a style issue during review; merging them into a single block can fail CI with `Imports are incorrectly sorted and/or formatted`.

Applied to files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

🔇 Additional comments (2)

autogpt_platform/backend/backend/blocks/orchestrator.py (1)

389-390: LGTM!

autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py (1)

299-309: LGTM!

---

Walkthrough

The SDK mode execution path now captures only the last assistant message that contains text and no tool-use blocks as the final response; intermediate assistant text is no longer accumulated into the finished output.

Changes

Final Response Text Extraction

Layer / File(s)	Summary
Final response text extraction mechanism autogpt_platform/backend/backend/blocks/orchestrator.py, autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py	Adds _select_final_answer_parts helper, a final_response_parts accumulator, collects assistant message text into per-message text_parts, updates final_response_parts via the helper when a message has no tool calls, and constructs the "finished" response from final_response_parts. Tests for the helper's behaviors are included.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

• Significant-Gravitas/AutoGPT#13180: Both PRs modify the SDK mode response handling in _execute_tools_sdk_mode.

Suggested labels

size/xl, platform/backend

Suggested reviewers

• Swiftyos
• ntindle

Poem

🐰 I sift the stream for the final line,
The middle bits I cast aside,
The last clear voice becomes the sign,
Silent tools don't steal the prize,
A focused answer, neat and spry.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 36.36% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely describes the main change: fixing orchestrator EXTENDED_THINKING mode to emit only the final answer instead of accumulating all text blocks.
Description check	✅ Passed	The description is directly related to the changeset, providing detailed context on the bug, the fix, edge cases, and testing approach for the orchestrator mode changes.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch zamilmajdy/orchestrator-built-in-accumulate-transcript

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[github-actions]

🔍 PR Overlap Detection

This check compares your PR against all other open PRs targeting the same branch to detect potential merge conflicts early.

🟢 Low Risk — File Overlap Only

These PRs touch the same files but different sections (click to expand)

• fix(orchestrator): forward credential input masks to tool node execution #13151 (mangod12 · updated 3d ago)

  • Shared files: autogpt_platform/backend/backend/blocks/orchestrator.py

• fix: propagate tool credentials to orchestrator when run from Library/AutoPilot (#13144) #13156 (paulxg12 · updated 3d ago)

  • Shared files: autogpt_platform/backend/backend/blocks/orchestrator.py

Summary: 0 conflict(s), 0 medium risk, 2 low risk (out of 2 PRs with file overlap)

---

Auto-generated on push. Ignores: openapi.json, lock files.

[codecov]

Codecov Report

❌ Patch coverage is 91.66667% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.54%. Comparing base (72e3995) to head (1e07f32).

Additional details and impacted files

@@            Coverage Diff             @@
##              dev   #13188      +/-   ##
==========================================
- Coverage   71.55%   71.54%   -0.01%     
==========================================
  Files        2221     2221              
  Lines      167661   167693      +32     
  Branches    17071    17076       +5     
==========================================
+ Hits       119964   119984      +20     
- Misses      44114    44123       +9     
- Partials     3583     3586       +3     

Flag	Coverage Δ
platform-backend	79.91% <91.66%> (+<0.01%)	⬆️
platform-frontend-e2e	30.82% <ø> (-0.09%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
Platform Backend	79.91% <91.66%> (+<0.01%)	⬆️
Platform Frontend	41.74% <ø> (-0.04%)	⬇️
AutoGPT Libs	∅ <ø> (∅)
Classic AutoGPT	28.43% <ø> (ø)

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@autogpt_platform/backend/backend/blocks/orchestrator.py`:
- Around line 389-391: The predicate that decides to treat text_parts as the
final "finished" answer currently returns list(text_parts) even for
blank/whitespace-only parts; modify the condition in the block that uses
text_parts, has_tool_calls, and current so it only returns list(text_parts) when
text_parts is non-empty, has_tool_calls is False, and at least one part contains
non-whitespace content (e.g., replace the existing check with one that includes
any(part.strip() for part in text_parts)); refer to the variables text_parts,
has_tool_calls and current to locate and update the condition.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: e3aee9d1-d551-4196-865e-24bf79ad357e

📥 Commits

Reviewing files that changed from the base of the PR and between 1f30b91 and a67da57.

📒 Files selected for processing (2)

• autogpt_platform/backend/backend/blocks/orchestrator.py
• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py

📜 Review details

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (10)

• GitHub Check: check API types
• GitHub Check: Seer Code Review
• GitHub Check: end-to-end tests
• GitHub Check: Analyze (python)
• GitHub Check: Analyze (typescript)
• GitHub Check: test (3.12)
• GitHub Check: test (3.11)
• GitHub Check: test (3.13)
• GitHub Check: Check PR Status
• GitHub Check: conflicts

🧰 Additional context used

📓 Path-based instructions (5)

autogpt_platform/backend/**/*.py

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

autogpt_platform/backend/**/*.py: Use Python 3.11 (required; managed by Poetry via pyproject.toml) for backend development
Always run 'poetry run format' (Black + isort) before linting in backend development
Always run 'poetry run lint' (ruff) after formatting in backend development

autogpt_platform/backend/**/*.py: Use poetry run ... command for executing Python package dependencies
Use top-level imports only — avoid local/inner imports except for lazy imports of heavy optional dependencies like openpyxl
Use absolute imports with from backend.module import ... for cross-package imports; single-dot relative imports are acceptable for sibling modules within the same package; avoid double-dot relative imports
Do not use duck typing — avoid hasattr/getattr/isinstance for type dispatch; use typed interfaces/unions/protocols instead
Use Pydantic models over dataclass/namedtuple/dict for structured data
Do not use linter suppressors — no # type: ignore, # noqa, # pyright: ignore; fix the type/code instead
Prefer list comprehensions over manual loop-and-append patterns
Use early return with guard clauses first to avoid deep nesting
Use %s for deferred interpolation in debug log statements for efficiency; use f-strings elsewhere for readability (e.g., logger.debug("Processing %s items", count) vs logger.info(f"Processing {count} items"))
Sanitize error paths by using os.path.basename() in error messages to avoid leaking directory structure
Be aware of TOCTOU (Time-Of-Check-Time-Of-Use) issues — avoid check-then-act patterns for file access and credit charging
Use transaction=True for Redis pipelines to ensure atomicity on multi-step operations
Use max(0, value) guards for computed values that should never be negative
Keep files under ~300 lines; if a file grows beyond this, split by responsibility (extract helpers, models, or a sub-module into a new file)
Keep functions under ~40 lines; extract named helpers when a function grows longer
...

Files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

autogpt_platform/backend/backend/blocks/**/*.py

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

autogpt_platform/backend/backend/blocks/**/*.py: Inherit from 'Block' base class with input/output schemas when adding new blocks in backend
Implement 'run' method with proper error handling in backend blocks
Generate block UUID using 'uuid.uuid4()' when creating new blocks in backend
Write tests alongside block implementation when adding new blocks in backend

autogpt_platform/backend/backend/blocks/**/*.py: For blocks handling files, use store_media_file() with return_format="for_local_processing" when processing with local tools (ffmpeg, MoviePy, PIL)
For blocks handling files, use store_media_file() with return_format="for_external_api" when sending content to external APIs (Replicate, OpenAI)
For blocks returning files, use store_media_file() with return_format="for_block_output" to enable auto-adaptation to execution context (workspace:// in CoPilot, data URI in graphs)
When creating new blocks, inherit from Block base class, define input/output schemas using BlockSchema, implement async run method, and generate unique block ID using uuid.uuid4()

Files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

autogpt_platform/backend/**/test/**/*.py

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

Use snapshot testing with '--snapshot-update' flag in backend tests when output changes; always review with 'git diff'

Files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py

autogpt_platform/{backend,autogpt_libs}/**/*.py

📄 CodeRabbit inference engine (AGENTS.md)

Format Python code with poetry run format

Files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

autogpt_platform/backend/**/test_*.py

📄 CodeRabbit inference engine (autogpt_platform/AGENTS.md)

Create a failing test first using @pytest.mark.xfail decorator (backend) when fixing a bug or adding a feature, then implement the fix and remove the xfail marker

Files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py

🧠 Learnings (15)

📚 Learning: 2026-02-05T04:11:00.596Z

Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 11796
File: autogpt_platform/backend/backend/blocks/video/concat.py:3-4
Timestamp: 2026-02-05T04:11:00.596Z
Learning: In autogpt_platform/backend/backend/blocks/**/*.py, when creating a new block, generate a UUID once with uuid.uuid4() and hard-code the resulting string as the block's id parameter. Do not call uuid.uuid4() at runtime; IDs must be constant across all imports and runs to ensure stability.

Applied to files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

📚 Learning: 2026-03-16T16:32:21.686Z

Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/pods.py:62-74
Timestamp: 2026-03-16T16:32:21.686Z
Learning: In autogpt_platform/backend/backend/blocks/, the Block base class execute() already wraps run() in a try/except to convert uncaught exceptions into BlockExecutionError/BlockUnknownError. Do not add per-block try/except in individual block run() methods, as this is not the established pattern (e.g., Gmail, Slack, Todoist blocks omit it). Only use explicit try/except within blocks that need to distinguish between success and error yield paths inside a generator (e.g., attachment blocks). This guidance applies to all Python files under autogpt_platform/backend/backend/blocks/ and similar block implementations; avoid duplicating error handling in run() unless a block requires generator-based branching.

Applied to files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

📚 Learning: 2026-04-23T12:55:26.122Z

Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12893
File: autogpt_platform/backend/backend/blocks/ayrshare/post_to_tiktok.py:24-24
Timestamp: 2026-04-23T12:55:26.122Z
Learning: Cost billing via the cost(*costs) decorator is applied at input-evaluation time (before a block’s run() executes). Therefore, mutating input_data inside run() will not change billing. When a block’s billing depends on a field plus URL/sniff-derived signals, treat the explicitly declared billing field (e.g., is_video) as the only billing source—set it correctly before run() (or in the code path that occurs before the decorator evaluates input_data). This should be checked for all blocks under autogpt_platform/backend/backend/blocks/ so billing signals are not mistakenly assumed to update during run().

Applied to files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

📚 Learning: 2026-02-26T17:02:22.448Z

Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12211
File: .pre-commit-config.yaml:160-179
Timestamp: 2026-02-26T17:02:22.448Z
Learning: Keep the pre-commit hook pattern broad for autogpt_platform/backend to ensure OpenAPI schema changes are captured. Do not narrow to backend/api/ alone, since the generated schema depends on Pydantic models across multiple directories (backend/data/, backend/blocks/, backend/copilot/, backend/integrations/, backend/util/). Narrowing could miss schema changes and cause frontend type desynchronization.

Applied to files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

📚 Learning: 2026-03-05T15:42:08.207Z

Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12297
File: .claude/skills/backend-check/SKILL.md:14-16
Timestamp: 2026-03-05T15:42:08.207Z
Learning: In Python files under autogpt_platform/backend (recursively), rely on poetry run format to perform formatting (Black + isort) and linting (ruff). Do not run poetry run lint as a separate step after poetry run format, since format already includes linting checks.

Applied to files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

📚 Learning: 2026-03-16T16:30:11.452Z

Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/threads.py:80-102
Timestamp: 2026-03-16T16:30:11.452Z
Learning: In autogpt_platform/backend/backend/blocks/ (and related blocks under autogpt_platform/backend/backend/blocks/), do not add try/except blocks around a block's run() method for standard error propagation. The block executor framework (backend/executor/manager.py) catches uncaught exceptions from run() and emits them on the 'error' output. Only add explicit try/except blocks when you need to control partial outputs in failure cases (e.g., certain outputs must not be yielded on error, as in attachment blocks). This is the standard pattern across the codebase; apply it broadly to blocks' run() implementations.

Applied to files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

📚 Learning: 2026-03-16T16:30:23.196Z

Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/pods.py:62-74
Timestamp: 2026-03-16T16:30:23.196Z
Learning: In any Python file under autogpt_platform/backend/backend/blocks, do not add a try/except around run() solely for standard error handling. The block framework’s _execute() in _base.py already catches unhandled exceptions and re-raises as BlockExecutionError or BlockUnknownError. If you yield ("error", message), _execute() raises BlockExecutionError immediately, so the error port will not propagate downstream. Reserve explicit try/except for scenarios where you must control partial output (e.g., attachment blocks that must skip yielding content_base64 on failure).

Applied to files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

📚 Learning: 2026-03-16T16:30:11.452Z

Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/threads.py:80-102
Timestamp: 2026-03-16T16:30:11.452Z
Learning: Do not wrap synchronous AgentMail SDK calls with asyncio.to_thread() in blocks under autogpt_platform/backend/backend/blocks (and across the codebase). The block executor runs node execution in dedicated threads via asyncio.run_coroutine_threadsafe (see manager.py around lines ~745-752 and ~1079). The existing pattern avoids using asyncio.to_thread for SDK calls inside async run() methods, so maintain that approach and do not add to_thread usage in these code paths.

Applied to files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

📚 Learning: 2026-03-16T16:35:40.236Z

Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/api/features/workflow_import.py:54-63
Timestamp: 2026-03-16T16:35:40.236Z
Learning: Avoid using the word 'competitor' in public-facing identifiers and text. Use neutral naming for API paths, model names, function names, and UI text. Examples: rename 'CompetitorFormat' to 'SourcePlatform', 'convert_competitor_workflow' to 'convert_workflow', '/competitor-workflow' to '/workflow'. Apply this guideline to files under autogpt_platform/backend and autogpt_platform/frontend.

Applied to files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

📚 Learning: 2026-03-19T15:10:50.676Z

Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12483
File: autogpt_platform/backend/backend/copilot/tools/test_dry_run.py:298-303
Timestamp: 2026-03-19T15:10:50.676Z
Learning: When using Python’s `unittest.mock.patch` in tests, choose the patch target based on how the imported name is resolved:
- If the code under test uses an **eager/module-level import** (e.g., `from foo.bar import baz` at module top), patch **the module where the name is looked up** (i.e., where it is used in the SUT), e.g. `patch("mymodule.baz")`.
- If the code under test uses a **lazy import** executed later (e.g., `from foo.bar import baz` inside a function/branch), patch **the source module** (e.g., `patch("foo.bar.baz")`) because the late `from ... import` will read the (potentially patched) name from the source module at call time.

For a concrete example: if `simulate_block` is imported inside an `if dry_run:` block in the SUT, then the correct test patch target is the source module path for `simulate_block` as it exists at call time (e.g., `patch("backend.executor.simulator.simulate_block")`), not the test file’s import location.

Applied to files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py

📚 Learning: 2026-03-31T15:37:38.626Z

Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12623
File: autogpt_platform/backend/backend/copilot/tools/agent_generator/fixer.py:37-47
Timestamp: 2026-03-31T15:37:38.626Z
Learning: When validating/constructing Anthropic API model IDs in Significant-Gravitas/AutoGPT, allow the hyphen-separated Claude Opus 4.6 model ID `claude-opus-4-6` (it corresponds to `LlmModel.CLAUDE_4_6_OPUS` in `autogpt_platform/backend/backend/blocks/llm.py`). Do NOT require the dot-separated form in Anthropic contexts. Only OpenRouter routing variants should use the dot separator (e.g., `anthropic/claude-opus-4.6`); `claude-opus-4-6` should be treated as correct when passed to Anthropic, and flagged only if it’s used in the OpenRouter path where the dot form is expected.

Applied to files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

📚 Learning: 2026-04-15T02:43:36.890Z

Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12780
File: autogpt_platform/backend/backend/copilot/tools/workspace_files.py:0-0
Timestamp: 2026-04-15T02:43:36.890Z
Learning: When reviewing Python exception handlers, do not flag `isinstance(e, X)` checks as dead/unreachable if the caught exception `X` is a subclass of the exception type being handled. For example, if `X` (e.g., `VirusScanError`) inherits from `ValueError` (directly or via an intermediate class) and it can be raised within an `except ValueError:` block, then `isinstance(e, X)` inside that handler is reachable and should not be treated as dead code.

Applied to files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

📚 Learning: 2026-04-22T11:46:04.431Z

Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12881
File: autogpt_platform/backend/backend/copilot/config.py:0-0
Timestamp: 2026-04-22T11:46:04.431Z
Learning: Do not flag the Claude Sonnet 4.6 model ID as incorrect when it uses the project’s established hyphenated convention: `anthropic/claude-sonnet-4-6`. This hyphen form is the intentional, production convention and should be treated as valid (including in files like llm.py, blocks tests, reasoning.py, `_is_anthropic_model` tests, and config defaults). Note that OpenRouter also accepts the dot variant `anthropic/claude-sonnet-4.6`, so either form may be tolerated, but `anthropic/claude-sonnet-4-6` should be considered the standard to match project usage.

Applied to files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

📚 Learning: 2026-04-22T11:46:12.892Z

Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12881
File: autogpt_platform/backend/backend/copilot/baseline/service.py:322-332
Timestamp: 2026-04-22T11:46:12.892Z
Learning: In this codebase (Significant-Gravitas/AutoGPT), OpenRouter-routed Anthropic model IDs should use the hyphen-separated convention (e.g., `anthropic/claude-sonnet-4-6`, `anthropic/claude-opus-4-6`). Although OpenRouter may accept both hyphen and dot variants, treat the hyphen-separated form as the intended, correct codebase-wide convention and do not flag it as an error. Only flag the dot-separated variant (e.g., `anthropic/claude-sonnet-4.6`) as incorrect when reviewing/validating model ID strings for OpenRouter-routed Anthropic models.

Applied to files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

📚 Learning: 2026-05-07T18:48:14.242Z

Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 13040
File: autogpt_platform/backend/backend/blocks/llm.py:0-0
Timestamp: 2026-05-07T18:48:14.242Z
Learning: In this repository, isort may split imports from the same module into separate blocks when some imports are aliased (e.g., `from module import X as Y`) and others are not. Preserve the two-block layout when it results from isort (such as keeping `from openai.types.chat import ChatCompletion as OpenAIChatCompletion` separate from non-aliased imports from `openai.types.chat`). Do not treat that split as a style issue during review; merging them into a single block can fail CI with `Imports are incorrectly sorted and/or formatted`.

Applied to files:

• autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
• autogpt_platform/backend/backend/blocks/orchestrator.py

🔇 Additional comments (2)

autogpt_platform/backend/backend/blocks/orchestrator.py (1)

1847-1853: LGTM!

autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py (1)

14-18: LGTM!

Also applies to: 216-298