Skip to content

fix(orchestrator): workflow planning is dead at every caller — un-awaited coroutines plus retired WorkflowTask field names #13730

Description

@mrveiss

Discovered while fixing #13699. That issue repaired the benchmark that exercises
Orchestrator.plan_workflow_steps; strengthening its assertions surfaced that the
method returned an empty plan for every caller. #13699 fixes the method itself.
Every remaining caller is still broken, in two independent ways.

1. Every production caller invokes the async planning API without await

Orchestrator.plan_workflow_steps and Orchestrator.classify_request_complexity are
both async def. Four call sites bind the coroutine object and then iterate it:

Call site Symptom
autobot-backend/orchestration/workflow_planner.py:113 for step in base_steps over a coroutine
autobot-backend/orchestration/workflow_planner.py:219 len(base_steps) over a coroutine
autobot-backend/services/workflow_automation/manager.py:75-76 both calls un-awaited
autobot-backend/services/advanced_workflow/step_generator.py:63 for i, base_step in enumerate(base_steps) over a coroutine

The benchmark repaired in #13699 is the only awaited caller in the tree:

$ grep -rn "await.*plan_workflow_steps" --include="*.py" .
./performance_benchmarks_test.py:366
./performance_benchmarks_test.py:401

Confirmed at runtime — pytest reports it directly:

RuntimeWarning: coroutine 'Orchestrator.plan_workflow_steps' was never awaited

2. Callers read field names the canonical task shape dropped in #6951

WorkflowStep has aliased autobot_shared.workflow.WorkflowTask since #6951 Phase 2A.
WorkflowTask exposes task_id and requires_approval. These sites still read the
retired names, so they would raise AttributeError even with the await added:

  • orchestration/workflow_planner.py:128 — step.id
  • orchestration/workflow_planner.py:133 — step.user_approval_required
  • orchestration/workflow_planner.py:227,230 — step.id, step.user_approval_required
  • services/workflow_automation/manager.py:83,85 — step.user_approval_required, base_steps[j].id

3. The e2e test for this method cannot fail

autobot-backend/orchestration/plan_steps_e2e_test.py has no assertions — it prints a
step count inside a try that catches every exception and prints the traceback. It
passed throughout the entire period the method returned nothing. This is the #13162
shape again: a green test file guarding a fully broken method.

Why it matters

Workflow planning from a chat request is dead end-to-end. create_workflow_from_chat_request
(services/workflow_automation/manager.py) and plan_workflow_steps_with_agents
(orchestration/workflow_planner.py) cannot produce a plan, and nothing in the suite
reports it.

Acceptance criteria

  • All four call sites await the planning coroutines.
  • All field reads use the canonical WorkflowTask names (task_id, requires_approval); no read of a name absent from the dataclass.
  • plan_steps_e2e_test.py asserts a non-empty plan per complexity level and no longer swallows exceptions — verified by a deliberate failure.
  • A test covers create_workflow_from_chat_request and plan_workflow_steps_with_agents returning real steps, so a regression to the empty-plan state fails CI.

Activity

  1. mrveiss commented on Aug 8, 2026

    @mrveiss
    OwnerAuthor

    Context: the planner-side half of this landed as #13731 (squash 2dd19bfe9, closing #13699). Orchestrator.plan_workflow_steps now constructs canonical WorkflowTask fields and returns a real plan.

    That makes this issue the only thing standing between the planner and working end-to-end workflow planning — every caller listed above is still un-awaited and still reads retired field names, so chat-request planning remains dead until this lands.

  2. github-actions commented on Aug 8, 2026

    @github-actions
    Contributor

    PR #13752 (merged to Dev_new_gui) references this issue with a close keyword.

    fix(orchestration): await the planning API and read canonical WorkflowTask fields (#13730)

    If this issue is fully resolved, close it manually. If work remains, no action is needed.

  3. mrveiss commented on Aug 8, 2026

    @mrveiss
    OwnerAuthor

    Merged and verified on Dev_new_gui.

    PR: #13752 · squash commit: 6a1d769e4

    Verified in base, not merge state

    $ git log --oneline -1 origin/Dev_new_gui
    6a1d769e4 fix(orchestration): await the planning API and read canonical WorkflowTask fields (#13730)
    
    # awaits on the planning API, per file
    2  services/workflow_automation/manager.py
    3  orchestration/workflow_planner.py
    1  services/advanced_workflow/step_generator.py
    
    # retired attribute reads remaining
    0  orchestration/workflow_planner.py
    0  services/workflow_automation/manager.py
    
    $ git ls-tree origin/Dev_new_gui --name-only autobot-backend/tests/orchestration/test_planning_caller_wiring.py
    autobot-backend/tests/orchestration/test_planning_caller_wiring.py
    
    $ python3 -m pytest tests/orchestration/test_planning_caller_wiring.py orchestration/plan_steps_e2e_test.py -q -p no:randomly
    6 passed in 1.22s
    

    Acceptance criteria

    • All four call sites await the planning coroutines. Three were in async def and needed only await; get_plan_summary was def calling an async API (running complexity.value and len(base_steps) against coroutines) and is now async def.
    • All field reads use canonical WorkflowTask names. step.id → task_id, step.user_approval_required → requires_approval, base_steps[j].id → .task_id. A grep for retired attribute reads returns nothing outside docstrings. Dict keys in workflow_planner.py are deliberately unchanged — they are that module's contract with create_plan_summary_for_approval (line 267), which reads step.get("user_approval_required"); only the attribute sources moved.
    • plan_steps_e2e_test.py asserts a non-empty plan per complexity level and no longer swallows exceptions. Also corrected a false premise in the old file: its four "complexity levels" were really two, because RESEARCH/INSTALL/SECURITY_SCAN all carry the value "complex" and are therefore Enum aliases of COMPLEX — it exercised COMPLEX three times. Now covers SIMPLE and COMPLEX explicitly.
    • A test covers create_workflow_from_chat_request and plan_workflow_steps_with_agents returning real steps. tests/orchestration/test_planning_caller_wiring.py drives all four callers with a real Orchestrator — a stub with invented attributes would have passed against the exact bug being fixed.

    Live impact this fixed

    Worse than this issue originally described. create_workflow_from_chat_request is reached by two HTTP routes (api/workflow.py:290, services/workflow_automation/routes.py:258). It returned None on every call, and api/workflow.py:292 maps a falsy id to HTTPException(500) — so every chat-driven workflow request was answering HTTP 500, not merely failing silently inside the orchestrator.

    Every fix proven load-bearing

    Each of the seven changes was reverted in isolation; all seven make the new tests fail, so none of them can pass against the broken code:

    Reverted Result
    await on manager.py plan call FAILED
    step.requires_approval → user_approval_required FAILED
    base_steps[j].task_id → .id FAILED
    await on workflow_planner.py:113 FAILED
    "id": step.task_id → step.id FAILED
    async def get_plan_summary → def ERROR
    await on step_generator.py:63 FAILED

    CI

    All 10 required contexts green or correctly skipped (migration-matrix / api-wiring — no migrations, no API surface). All 13 python-suite entries (12 shards + aggregate) SUCCESS, so not a pytest-split empty-shard false green. 995 tests passed locally across every touched area, re-run after a mid-flight rebase onto #13734.

    Carried forward — #13751

    WorkflowPlanner.plan_workflow_steps_with_agents and get_plan_summary have zero callers. Both were corrected and are now directly tested, but neither is reachable from any production path — agent-assigned planning and plan preview remain unwired. Their unreachability is also why two of the four bugs fixed here survived so long: no runtime signal can point at code nothing executes. Per the standing rule they were kept and fixed, not removed.

  4. mrveiss commented on Aug 9, 2026

    @mrveiss
    OwnerAuthor

    Follow-ups filed from discoveries made while fixing this: #13808 (finish #6951 Phase 3 — the same-named-different-shape WorkflowStep that manager.py both reads and constructs is the direct source of the retired-field-name bugs fixed here), #13806, #13807.

  5. mrveiss commented on Aug 9, 2026

    @mrveiss
    OwnerAuthor

    Verified the downstream effect of this fix rather than assuming it, and filed #13809: the path is now live, but every chat request yields the same three echo 'Executing: <action>' steps because plan_workflow_steps is a fixed skeleton and the command fallback cannot match its action names. No injection risk (user text never reaches the command — checked with a hostile request), but the endpoint now fails silently where it previously failed loudly with a 500. Worth closing that gap.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions