Skip to content

Step results are binary: no partial-success envelope, so a retry re-runs work that already succeeded #468

Description

@Masterplanner25

Summary

A workflow step result is binary: it succeeded or it failed. There is no shape for
"3 of 5 sub-units applied; here are the 2 that did not and why" — so a retry
re-attempts the whole step, including the parts that already worked.

Current state

Run status is a closed set of seven (src/nodus_lang_workflow/models.py):
pending, running, waiting, retry_scheduled, completed, failed,
dead_lettered. Nothing between completed and failed.

The consequence is on the retry path, not the reporting path. When a step defers
(retry_scheduled) the sweeper resumes it and it re-runs whole. If the step wrote
three of five files, the retry writes all five again — correct only because the
writes happen to be idempotent, which the runtime does not know and cannot check.

What to add

A partial terminal shape carrying per-sub-unit outcomes, and a retry that consumes
it:

{status: "partial", applied: [...], failed: [{unit: ..., error: ...}]}

with the retry re-attempting only failed. Sketch of the decisions this needs:

  • Is partial a run status or a step result field? Adding an eighth run status
    touches every store, the CLI (nodus workflow runs|inspect|dead-letters), the
    HTTP endpoints, and any consumer switching on status. A result-map field is far
    cheaper and probably sufficient — the run itself is still retry_scheduled or
    completed.
  • What is a sub-unit? Only the step author knows. This has to be author-declared
    (the step returns the envelope) rather than runtime-inferred; the runtime's job is
    to carry it across the defer/resume boundary and hand failed back on resume,
    which is the part a step cannot do for itself.
  • Interaction with @exactly_once. The effect ledger currently keys a whole
    action. Partial application means an action can be half-recorded, and the ledger
    needs to say what it will do about that before this ships — otherwise the two
    features disagree about what "done" means.

Provenance

C:\codev\Aider research\AIDER_ON_AINDY_RUNTIME_PORTABILITY_ANALYSIS.md §9,
primitive B2 — "Partial-success envelope — {status: "partial", applied, failed} with per-sub-unit ledger records", tagged New. Also
AIDER_AINDY_LENS_AUDIT.md §5, where it is the second absorb item for the runtime:
"Aider's '3 of 5 hunks applied; here are the 2 that failed and why' is exactly the
structured failure payload a retry path wants."

The docs file it against aindy-runtime's syscall envelope, which is likewise binary
(syscall_dispatcher.py:22). Nodus has the same shape and the same gap, and the
Nodus half is the one that owns step retry, so it is filed here on its own merits
rather than as a downstream consequence.

Lowest priority of the four issues from this cross-check: it only bites steps whose
work divides into independently-failing units, and the workaround — a step that
writes its own progress into workflow state and skips completed units on resume —
already works today, since goal iterations resume the same graph
(src/nodus/vm/vm.py:1433-1448).

Affected versions

v5.0.4 (current when filed). Not a regression.

Activity

  1. added this to the v5.0 milestone on Aug 18, 2026
  2. Masterplanner25 commented on Aug 18, 2026

    @Masterplanner25
    OwnerAuthor

    Filed together from one cross-check of the four C:\codev\Aider research documents against 5.0.4 (AIDER_AINDY_LENS_AUDIT.md, AIDER_ON_AINDY_RUNTIME_PORTABILITY_ANALYSIS.md, ACCURACY_CHECK_vs_aindy-runtime_2.4.0.md; AIDER_ARCHITECTURAL_AUDIT.md contains no Nodus claims). The set:

    Ranked: #467 first, then #466, then #465, then #468.

    Nothing else the documents raised is Nodus work — the remaining corrections are aindy-runtime pinning nodus-lang==5.0.1 and needing 5.0.4 (#411 annotation forgery, #185 shared memory store, #390 workflow ownership all land in 5.0.2/5.0.3, and 5.0.3 alone breaks nodus-sdk).

  3. removed this from the v5.0 milestone on Aug 25, 2026
  4. Masterplanner25 commented on Aug 26, 2026

    @Masterplanner25
    OwnerAuthor

    Decided — superseded by #480. Recommending closure

    Taken with the other seven open workflow-DSL questions in one session. Decisions:
    docs/design/workflow-dsl/00-cluster-decisions.md (D6), PR #610.

    Read next to #480, this is the same requirement at a finer granularity.

    Those are one requirement, and neither issue says so because they were filed from
    different corpora four apart.

    Why #480's answer is the better mechanism

    The per-unit machinery already exists — one level up. TaskNode carries
    max_retries, retry_delay_ms, timeout_ms, cache, cache_key, on_states,
    when and allow_failure per node. #480's decided model (Airflow's mapped
    node: the graph does not grow, a declared node's cardinality is discovered at run
    time) expands one node to N instances, and each gets all of those for free.

    So "retry only the two that failed" becomes the ordinary retry path rather than a
    new concept, and three of this issue's own open questions dissolve rather than
    being answered:

    • "Is partial a run status or a step result field?" Neither. This issue is
      right that an eighth run status "touches every store, the CLI, the HTTP
      endpoints, and any consumer switching on status" — and a mapped node needs none
      of it. Worth adding that there are two closed vocabularies, not one: seven
      run statuses in nodus_lang_workflow/models.py and seven task statuses in
      TASK_STATUSES, disjoint, the latter pinned by
      tests/test_status_vocabulary.py.
    • "What is a sub-unit?" The item in the list the producer returned. Still
      author-declared, as this issue correctly insists — just declared as a fan-out
      rather than as an envelope shape.
    • "Interaction with @exactly_once." This is the one that mattered most and
      the one a mapped node removes entirely. The issue's own framing — "the two
      features disagree about what 'done' means"
      — is a consequence of introducing a
      second definition of done. Each mapped instance is a whole action; the ledger
      keys it exactly as it keys any other. No disagreement to resolve.

    What survives, unchanged

    The workaround this issue documents stays valid and D5 does not take it away.
    Below mapped-node granularity — three of five hunks inside one file edit — the
    sub-unit is not worth a durable record, and "a step that writes its own progress
    into workflow state and skips completed units on resume"
    already works. This
    issue says so itself.

    What would reopen this

    A case where the sub-units genuinely cannot be nodes and the state
    workaround genuinely fails. Neither this issue nor #480 has produced one — Aider's
    "3 of 5 hunks applied" is precisely the sub-node granularity the workaround
    covers.

    If that case appears, reopen against a mapped-node baseline rather than against
    today's, because the argument changes: the question would then be whether a
    node needs a partial envelope, not whether a step does.

    Recorded on #480 as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions