Add multi-agent safety and sg seed repository bootstrapping#6
Open
karstom wants to merge 3 commits into
Open
Conversation
…(v0.3.0) New `sg` bin in the mcp package. `sg seed` mines a repo's git history and working tree into a draft memory graph: reverts and fix commits become Regressions, ADRs and merge bodies become Decisions, rule comments and rule-shaped test names become Invariants, TODO markers and high-churn files become Watchlists, and top-level structure becomes Components. Deterministic, offline, no API key. - Extractor interface with the five Tier-1 deterministic extractors; an LLM enrichment seam (enrich.ts) is designed but intentionally unimplemented - Edge inference: CONTAINS ownership, SUPERSEDED_BY/CAUSES revert pairs, RELATES_TO for shared issues and co-change coupling (both edge types newly documented in HOW_TO_UPDATE.md) - Provenance on every seeded node (**Provenance:** commits/locations, **Seeded:** extractor, confidence, content hash), round-tripping through the parser and MCP tools - Review gate: summary + draft bundle + interactive confirm (--yes / --dry-run); quality controls via --min-confidence, --max-per-type, dedupe - Idempotent merges: stable content-derived IDs, hand-edited seeded nodes are never overwritten (conflict reported), high-water mark in core/.seed_state.json - 17 fixture-repo tests (temp git repos, never the live repo); full 5k-commit window mines in <1s - Fix: importing mcp/src/index.ts no longer attaches the MCP server to stdio, so node --test exits; consistency_check.sh now ignores fenced template examples Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ADCdFZ7mid9wnNeexHJp5w
Concurrent writers — parallel sessions, worktree subagents, or a whole team — stress the graph in ways a single agent never does. Three fixes: - Atomic writes + a per-graph advisory lock across every read-modify-write tool call, closing the lost-update race on REGRESSED_N_TIMES and torn reads on shared checkouts. Stale locks (crashed holder) are reclaimed by timestamp; the lock is always released, including on error. - graph_index.md's Quick Index is now derived, not hand-appended. New `sg reindex` command and `simplegraph_reindex` tool rebuild it deterministically (sorted, order-independent) — the intended way to resolve index merge conflicts. add_node/archive regenerate it automatically; the manual update_index step is gone (kept as an alias). - Optional Author/Session fields record which agent/session created a node, for arbitrating conflicts after a merge. Stamped from add_node args or SIMPLEGRAPH_AUTHOR/SIMPLEGRAPH_SESSION; omitted when unset, so existing nodes and seed output are unchanged. - consistency_check.sh now fails on duplicate node IDs — the collision two branches can create without a git conflict. 14 new tests (fsutil, reindex, attribution) plus real concurrency verification; all 38 pass. Node format only grows optional fields; seed output is byte-identical to 0.3.0. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ADCdFZ7mid9wnNeexHJp5w
Completes the multi-agent work (v0.4.0) with the two remaining items, kept deliberately minimal: Propagation across branches - Ship core/.gitattributes (and shared/) marking the append-only node list files merge=union — a built-in git driver, no config. Two branches that each appended a node now merge cleanly instead of conflicting on every add. graph_index.md is excluded on purpose (regenerate with sg reindex); components/ is one-node-per-file. setup.sh installs and refreshes these. - Documented the propagation convention: a node reaches other agents only when its branch merges, so commit graph updates with the code and land them promptly; fetch before parallel work. Trust boundary for the shared graph - The server was already read-only against shared/, so promotion is a human-only act; made that the documented control and prescribed extra review for shared-graph PRs. - consistency_check.sh is now shared-aware: auto-detects a sibling shared/ (or --shared <dir>), validates IDs and edges across both graphs so cross-graph edges resolve instead of reading as broken, and warns when a shared node has no attribution (Author/Provenance/Seeded) — an org-wide rule with no traceable source. Verified: union merge resolves a two-branch append that conflicts without the attribute; shared-aware check confirmed on cross-graph edges, a cross-graph duplicate ID, a broken edge, and an untraced shared node; all 38 MCP tests still pass. Inert when no shared/ graph is present. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ADCdFZ7mid9wnNeexHJp5w
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This release hardens simplegraph for concurrent writers and adds a deterministic CLI tool to bootstrap memory graphs from repository history. Two major features:
sg seedCLI — mines git history and working tree to auto-populate the graph with Regressions (reverts, repeated fixes), Decisions (merge bodies, ADRs), Invariants (emphatic comments, test names), Watchlists (TODOs, churn), and Components (directory structure).Key Changes
Concurrency Hardening
fsutil.ts): All graph file writes now use temp-file-plus-rename to prevent readers from observing half-written nodes.fsutil.ts): Read-modify-write operations (add_node,update_node,archive_regression) hold a per-graph lock across the entire operation, closing the lost-update race where two sessions both readREGRESSED_N_TIMES = Nand both writeN+1.reindex.ts):graph_index.md's node-type rows are now regenerated deterministically from the node files rather than hand-appended, making concurrent merges conflict-free. Task Routing section remains hand-authored.parser.ts,index.ts): Nodes now carry optionalauthorandsessionfields for tracing which agent created them.Repository Bootstrapping (
sg seed)seed/cli.ts):sg seed [PATH]mines a repository with options for history window, confidence thresholds, node-type filters, and output location.seed/mine.ts): Extracts commits, file changes, and working-tree inventory from git, building the context for extractors.seed/extractors/):history.ts: Regressions from reverts and repeated-fix clusters; Decisions from merge bodies and ADR documents.worktree.ts: Invariants from emphatic comments and rule-shaped test names; Watchlists from TODO/FIXME markers and high-churn files; Components from directory structure.seed/bundle.ts): Runs extractors, applies quality controls (confidence floor, per-type caps, cross-extractor dedupe), and infers edges.seed/merge.ts): Idempotent merge of draft nodes into graph files, detecting hand-edits and conflicts.seed/ids.ts): Content-derived node IDs survive cosmetic changes (e.g., a TODO moving between files) for reliable dedupe.seed/types.js): Shared types for the mining pipeline (DraftNode, Provenance, Extractor, etc.).Testing & Documentation
seed/seed.test.ts): 20+ tests covering extraction, provenance, idempotency, hand-edit conflicts, dedupe, and edge integrity against throwaway fixture repos.multiagent.test.ts): Validates atomic writes, graph locking, and index regeneration.scripts/consistency_check.sh): Enhanced to detect duplicate IDs and validate edges across core/ and shared/ graphs..gitattributes): Union-merge for node list files so concurrent additions don't conflict.Integration
handleAddNode,handleUpdateNode, and file writes now useatomicWriteFileSync.simplegraph_add_nodeandsimplegraph_archive_regressionnow automatically regenerate the Quick Index.https://claude.ai/code/session_01ADCdFZ7mid9wnNeexHJp5w