Sequential cascade engine: rebase and re-run on upstream changes

[chanakyav]

Parent Epic

Part of #52 — Sequential stacked PR chains

Summary

When an upstream task in a stacked PR chain pushes new commits (e.g., from a FIX cycle after reviewer feedback, or user restarting a task), automatically rebase downstream tasks and re-run their agents. Cascades propagate sequentially down the chain.

Uses standard git checkout + git rebase for branch operations (no worktrees).

Context

In a stacked chain, task #2's branch is based on task #1's branch. When task #1 gets new commits, task #2's branch is stale. Without cascade, the user must manually rebase and re-run every downstream task — defeating the purpose of the chain.

Simplified Design (Sequential Only)

V1 is sequential:

• Tasks run one at a time, so there's no "pause mid-agent" complexity
• Cascade happens after all tasks in the chain are COMPLETE
• When a user restarts an upstream task, downstream tasks are invalidated and re-run in order

Flow

1. Chain completes: Task 1 (COMPLETE) → Task 2 (COMPLETE) → Task 3 (COMPLETE)
2. Human reviewer gives feedback on PR Add fix-ci mode: fix CI failures from PR check annotations #1
3. User runs autopilot restart <task1_id> — task 1 re-enters the review-fix loop
4. Task 1 pushes new commits, reaches COMPLETE again
5. Cascade engine detects: task 1's branch HEAD differs from what task 2 was based on
6. For task 2: git checkout <task2_branch> && git rebase origin/<task1_branch>, re-run agent with same prompt + "upstream changed" context
7. Task 2 completes → cascade to task 3 (same process)

Changes

chain.py (new module, created in #57)

• check_chain_staleness(epic_id) — for each task in chain, compare stored upstream_sha vs current upstream HEAD
• rebase_task(task_id, upstream_branch) — git checkout <branch> && git fetch origin && git rebase origin/<upstream_branch>
  • If clean: force-push, return True
  • If conflicts: abort rebase (git rebase --abort), return False with conflict info
• run_cascade(epic_id, starting_from_task) — iterate downstream tasks, checkout + rebase + re-run each sequentially

persistence.py

• upstream_sha column (from Schema: parent_task_id, epic_id, base_branch, upstream_sha columns #55) — last-known HEAD SHA of the upstream branch when task started
• get_chain_tasks(epic_id) returns ordered list (from Schema: parent_task_id, epic_id, base_branch, upstream_sha columns #55)

prompts.py

• Add cascade_prompt(original_prompt, upstream_changes_summary) — tells agent: "The upstream code changed. Review your implementation against the new base and adjust if needed."
• Add rebase_conflict_prompt(original_prompt, conflict_files) — tells agent to resolve rebase conflicts in listed files

cli.py

• Modify cmd_restart(): when restarting a task that has downstream dependents (part of a chain), after the task completes, trigger run_cascade() for downstream tasks

Design Decisions

• Always re-run after rebase: even a clean rebase may produce logically incorrect code. The agent reviews the implementation against the new upstream.
• Max cascade attempts per task: 2. If a downstream task can't stabilize after 2 re-runs, mark as needing manual attention.
• Sequential propagation: cascade task 2, wait for COMPLETE, then cascade task 3. No parallel cascade.
• Conflict resolution: agent gets conflict files in prompt. If it can't resolve, task pauses with clear status.

Depends On

• Schema: parent_task_id, epic_id, base_branch, upstream_sha columns #55 (schema: upstream_sha)
• base_branch support for stacked PRs #54 (base_branch support)
• autopilot chain command with --epic and --issues flags #57 (chain command and chain.py module)

Testing

• Upstream task restarts and pushes new commits → downstream detects SHA mismatch
• Clean rebase → agent re-runs with cascade context prompt
• Rebase conflict → agent gets conflict resolution prompt
• Cascade chain: task 1 → task 2 → task 3 propagates correctly
• Max cascade attempts (2) respected — then task pauses
• No cascade triggered when upstream SHA hasn't changed
• Upstream SHA stored and updated correctly in persistence

[IlyasFardaouix]

I think I see the issue here. It looks like the cascade engine is not properly handling the case where a downstream task is in the middle of an agent run when the upstream task changes. The rebase_onto_upstream method is called with a worktree_path that might not be the current working directory, which could lead to unexpected behavior. Have you considered adding a check to ensure that the worktree_path is the current working directory before attempting the rebase? Also, could you clarify what happens if the rebase conflicts and the agent is unable to resolve them after two attempts? Does it mark the task as needing manual attention, but what happens to the downstream tasks?

[chanakyav]

Architecture Note: File Placement

All cascade logic goes in chain.py alongside the sequential runner (from #57). Functions:

• check_chain_staleness(epic_id) — compare stored upstream_sha vs HEAD
• rebase_task(task_id, upstream_branch) — git rebase in worktree
• run_cascade(epic_id, from_task) — iterate downstream, rebase + re-run

Cascade prompts (cascade_prompt, rebase_conflict_prompt) go in prompts.py.

This keeps the cascade engine co-located with the chain runner in a single focused module.