Skip to content

#476: the two halves of a run share a lifecycle - #567

Merged
Masterplanner25 merged 1 commit into
mainfrom
476-run-store-lifecycle
Aug 25, 2026
Merged

Masterplanner25 merged 1 commit into
mainfrom
476-run-store-lifecycle

Conversation

@Masterplanner25

Copy link
Copy Markdown
Owner

Summary

Fixes #476 — options (1) and (2) of its fix direction, both directions of each.

Reproduced on main first: the issue's repro (rm -rf .nodus/workflow_framework/runs — the cleanup CLAUDE.md described as safe) made a live waiting run permanently unresumable behind Workflow run '…' not found, with its graph state and checkpoint sitting on disk.

Symmetric cleanup (option 1), both directions:

  • nodus workflow cleanup now deletes the run record with the graph state, reported as run_records_removed in its output. Guard: if the record says the run is still live (the administrative mark_waiting shape — a waiting record over a completed snapshot), it is not cleanup's to take without --force.
  • The store's opt-in max_terminal_runs cap now prunes the graph state and checkpoint with the record it deletes — a host that caps records wants the run gone, not half of it.
  • Stores gained delete_run(run_id) -> bool — a concrete no-op default on the WorkflowStore ABC (an abstract method would break every host store implementation to gain a capability only cleanup uses), overridden by LocalWorkflowStore and SQLiteWorkflowStore. The SQLite version goes through a new _exec_rowcount helper: the existing #516 source-assertion (conn.execute only inside the cursor-owning helpers) caught my first cut calling conn.execute directly, which is that test doing exactly its job.

Honest diagnosis (option 2): a resume whose record is missing while load_graph_state finds state now says so — "has graph state under .nodus/graphs but no run record in the workflow store — the two halves of the run were cleaned independently" — with category: "run_record_missing", plus the recovery hint (workflow cleanup --force for the orphaned state). An id with neither half still says not found, pinned by a falsifiability control.

Option (3) (one owner) not taken — recorded on the issue as the long-term shape, with #380 the natural companion.

CLAUDE.md's "safe to rm -rf" note is corrected; it was this issue's repro.

Test plan

  • tests/test_run_store_lifecycle.py (# closes: #476): cleanup removes both halves and reports both lists; the issue's rm-the-records repro now yields the honest diagnosis instead of not found; unknown ids still say not found; the record cap prunes graph state with the record (timestamp-tie made deterministic); SQLite delete_run round-trip.
  • 338 store/cleanup/workflow/resume/runner/framework/wait tests pass (including the SQLiteWorkflowStore leaks cursors: the store depends on CPython refcounting to commit #516 cursor-discipline source assertion, now green); ruff + mypy clean; closed-issues gate passes for all eight [Unreleased] issues.

🤖 Generated with Claude Code

https://claude.ai/code/session_01QBC6mwW25PMi72GqL9Q9vX

A durable run is split across .nodus/graphs/ (graph state + checkpoint)
and the workflow store (run record), and nothing kept them in step:
workflow cleanup removed graph state and left records accumulating
forever; the store's opt-in max_terminal_runs cap deleted records and left
graph state orphaned; and a resume whose record was gone -- while the
state sat on disk -- reported "not found" (reproduced on main: the
issue's rm -rf on the runs directory made a live waiting run permanently
unresumable behind that message).

Now: cleanup deletes the run record with the graph state, unless the
record says the run is still live and --force was not given (the
mark_waiting administrative shape is live state, not cleanup's to take);
the record cap prunes graph state and checkpoint with the record; stores
gained delete_run -- a concrete no-op default on the WorkflowStore ABC so
host implementations keep working, overridden by Local and SQLite (via a
new _exec_rowcount helper, keeping the #516 cursor discipline); and the
missing-record resume names the real situation with
category "run_record_missing".

CLAUDE.md's "rm -rf .nodus/workflow_framework/runs is safe" note is
corrected -- it was this issue's repro.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QBC6mwW25PMi72GqL9Q9vX
@Masterplanner25
Masterplanner25 merged commit f6e44e7 into main Aug 25, 2026
4 checks passed
@Masterplanner25
Masterplanner25 deleted the 476-run-store-lifecycle branch August 25, 2026 15:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A run is split across two stores with no shared lifecycle: cleanup orphans state in both directions

1 participant