Capture Claude/Codex harness events as Basic Memory producer envelopes #997

Activity

1. claude commented on Jun 13, 2026

   

   claudebot

   on Jun 13, 2026 – with Claude

   Last edited by claude

   Contributor

   More actions

   Claude finished @phernandez's task in 1m 55s —— View job

   ---

   Issue Triage

   • Load tools
   • Explore existing plugin/hook structure
   • Check for duplicate issues
   • Classify issue type, priority, component, complexity
   • Apply labels
   • Provide triage summary

   ---

   Classification

   Dimension	Assessment
   Type	Feature request — new functionality
   Priority	Medium — high strategic value but no existing functionality is broken
   Component	Plugins (plugins/claude-code/, plugins/codex/), with cross-cutting impact on integrations/openclaw/
   Complexity	Complex — architectural work across multiple plugin packages

   Labels applied: enhancement

   ---

   No Duplicates Found

   Searched for "harness WAL", "producer envelope", "SPEC-55", and "hook events" — no prior issues. This is a new initiative.

   ---

   Current State of the Harness

   The existing hook landscape is minimal but real:

   Claude Code plugin (plugins/claude-code/hooks/):

   • SessionStart → session-start.sh — briefs Claude from the knowledge graph
   • PreCompact → pre-compact.sh — checkpoints the session before compaction

   Codex plugin (plugins/codex/hooks/):

   • Same two hooks, with matchers (startup|resume|compact and manual|auto)

   OpenClaw integration (integrations/openclaw/hooks/capture.ts):

   • Already has a post-turn capture handler that indexes conversation turns as notes via client.indexConversation()
   • Filters by minimum char count, extracts last user+assistant turn — this is a working example of the producer side

   The issue is asking for a more structured, normalized version of this pattern shared across Claude Code and Codex (not just OpenClaw), with defined envelope types, idempotency, opt-in config, and redaction.

   ---

   Complexity Assessment

   This is a complex feature for the following reasons:

   1. Shared helper across two plugin ecosystems: Claude Code uses shell scripts + bash; Codex uses Python hooks (pre-compact.py, session-start.py) and shell. A shared envelope schema that works across both (and possibly OpenClaw's TypeScript) requires careful design.

   2. Open architecture questions: The issue itself lists ~7 open design questions (inbox vs. direct write, which hooks are stable, where raw retention lives, etc.) — these need resolution before implementation.

   3. Privacy/redaction infrastructure: Deny rules for paths, tools, env-like values, large payloads is non-trivial and easy to get wrong.

   4. New event types: 12 proposed normalized event types, most of which have no current hook support (e.g. PostToolUse, file_changed, test_ran) — availability depends on what Claude Code and Codex actually expose.

   5. SPEC-55 / SPEC-61 dependencies: This is described as the "producer side" of SPEC-55 and feeds SPEC-61 routines. Implementation correctness depends on those specs being stable.

   ---

   Recommended Scope for v0

   Based on the existing code, a minimal v0 could be:

   1. Share the envelope shape — a simple JSON schema (or markdown spec) defining the normalized envelope fields (source, session_id, turn_id, cwd, project, timestamp, hook_name, payload_summary)
   2. Hook PreCompact only — both plugins already have this; extend the checkpoint scripts to emit a structured envelope alongside the existing session note
   3. Reuse the OpenClaw capture.ts pattern — adapt it for the shell/Python hooks
   4. Idempotency key = sha256(session_id + hook_name + timestamp) — write to a note only if that key isn't already present

   This deferral of PostToolUse and the more exotic event types (file_changed, test_ran) is consistent with the "V0 should only capture events exposed through supported harness hooks" note in the issue.

   ---

   Open Questions to Resolve Before Implementation

   The issue lists several — the two most blocking are:

   1. Inbox vs. direct write: Direct write through existing MCP calls is simpler for v0; an inbox adds durability at the cost of a new persistence layer. Recommend direct write for v0.
   2. Hook availability: Claude Code exposes PostToolUse — confirm whether Codex does, and whether it's stable enough to depend on.
2. claude

   added

   enhancementNew feature or requestNew feature or request

   on Jun 13, 2026
3. sourrris commented on Jun 13, 2026

   

   sourrris

   on Jun 13, 2026

   Contributor

   More actions

   I am working on this now.
4. hegu-1 commented on Jun 30, 2026

   

   hegu-1

   on Jun 30, 2026

   More actions

   This producer-envelope direction is very close to how I’d expect a personal memory vault to ingest agent work traces.

   One design distinction I’d make explicit: the raw harness stream should not itself become “memory.” It is trace material. Durable memory should be a promoted artifact derived from that trace.

   A useful pipeline shape could be:

   raw_event          = append-only, short/medium retention, replay/debug
   tracelet           = normalized visible event with provenance
   session_note       = compressed continuity summary
   tool_ledger        = what was called, why, and what changed
   candidate_memory   = proposed durable fact/decision/preference
   vault_memory       = user-owned, promoted, editable markdown source-of-truth
   

   That keeps the vault from becoming a transcript landfill while still preserving enough evidence to audit how a memory was formed.

   For v0, I’d consider making the envelope carry a few fields that support later promotion:

   source_agent
   source_tool
   project_hint
   visibility: private | project | public
   event_type
   provenance_ref
   redaction_status
   promotion_status: raw | summarized | candidate | accepted | rejected
   

   The promotion_status field matters because agents should be able to propose memory, but not silently turn every observed session detail into durable user memory.

   For personal vault usage, the ideal contract is: raw traces are available for audit; generated summaries are replaceable; authored/promoted vault notes are the source of truth. That distinction is what lets memory stay user-owned across Claude, Codex, and future tools.

   I’ve been exploring a small Personal AI OS / Memory Vault pattern around this same boundary: continuity across tools without making any single agent runtime the owner of memory.

   Related public projects:

   • https://github.com/hegu-1/personal-memory-vault-starter
   • https://github.com/hegu-1/personal-ai-os
5. phernandez commented on Jul 15, 2026

   

   phernandez

   on Jul 15, 2026

   MemberAuthor

   More actions

   Direction update after two implementation rounds (#998, #1064), a survey of prior art (claude-hook-utils, codex-honcho, graphify, recall), and re-alignment with SPEC-55. The architecture in SPEC-55 stands; one call changes: the async producer front door moves into the bm CLI itself. Recorded as a dated revision on SPEC-55 (2026-07-15).

   The revised shape:

   • New bm hook <event> command group in core (basic_memory/hooks/): parses the harness hook JSON on stdin (per-harness adapters for Claude Code and Codex), builds the producer envelope, applies redaction, and writes it to the SPEC-55 inbox (~/.basic-memory/inbox/, atomic tmp+rename). No daemon required to capture — envelopes accumulate from day one.
   • Plugins ship zero logic: hook manifests plus one-line shims. bm resolution order: BM_BIN → basic-memory/bm on PATH (preferred, keeps the hook consistent with the user's MCP server) → uvx "basic-memory>=<floor>" as the zero-install default, with the floor bumped by release tooling. uv is the documented per-platform prerequisite the plugin checks for. No vendoring, no shell heredocs — hook logic gets normal tests, typing, and releases in core.
   • Envelope contract additions: actor, caused_by, and promotion_status (raw → summarized → candidate → accepted/rejected) so raw trace never silently becomes durable memory. Projected artifacts carry created_by + caused_by_event provenance alongside the required [source] observation.
   • Phase 2a scope: capture for session-start / pre-compact, an interim deterministic projector (bm hook flush — SessionNote skeletons and ToolLedger entries, no LLM), and bm hook install / remove / status with surgical, ownership-tagged config merging per harness. Execution is fail-open (never disrupt a session); the capture gate is fail-closed (captureEvents must be boolean true). The SPEC-55 ingestion worker supersedes flush when the SPEC-54 daemon lands.

   This supersedes PRs #998 and #1064. The envelope internals proven there — the contract shape, idempotency keying, recursive redaction, and the fail-closed capture gate — carry into basic_memory/hooks/ with commit credit to @sourrris. Implementation PR to follow.
6. phernandez

   closed this as completedin #1070on Jul 20, 2026
7. phernandez

   added this to the v0.23 milestone on Aug 11, 2026