Skip to content

Feature: Coding agent sidecar — watch session transcripts, build knowledge graph #669

Description

@bm-clawd

SPEC: Coding Agent Sidecar

Summary

A file-watcher-based sidecar that observes coding agent session transcripts and ingests them as structured Basic Memory notes. No hooks, no plugins, no coupling to agent internals. Just watches files and builds a knowledge graph.

Problem

Coding agents (Claude Code, Codex, Cursor, OpenCode) forget everything between sessions. Solutions like Letta's claude-subconscious inject themselves into the agent's runtime — fragile, vendor-specific, opaque. CLAUDE.md is a single flat file with no structure or search.

Users need persistent, searchable, cross-agent memory that:

  • Survives across sessions
  • Works across different coding agents
  • Is human-readable and editable
  • Builds structured knowledge, not just raw logs

Approach

Watch session files → Extract knowledge → Write BM notes → Available via MCP

The sidecar never touches the coding agent. It reads session transcript files from disk, runs an extraction agent over them, and writes structured Basic Memory notes. The coding agent picks up that knowledge on its next session via BM's MCP server (already installed).

┌──────────────┐     ┌──────────────────┐     ┌──────────────┐
│ Claude Code  │     │   BM Sidecar     │     │ Basic Memory │
│ Codex / etc  │     │                  │     │              │
│              │     │  File Watcher    │     │  Knowledge   │
│  writes to:  │────▶│  ↓               │────▶│  Graph       │
│  ~/.claude/  │     │  Extract Agent   │     │              │
│  sessions/   │     │  ↓               │     │  MCP Server  │
│              │     │  Write BM Notes  │     │  ↑           │
│              │◀────────────────────────────────┘           │
│  (reads via  │     │                  │     │              │
│   MCP tools) │     │                  │     │              │
└──────────────┘     └──────────────────┘     └──────────────┘

Session File Locations

Agent Session Path Format
Claude Code ~/.claude/sessions/ JSONL (messages with role, content, tool_use)
Codex CLI ~/.codex/sessions/ JSON (conversation history)
Cursor Varies (workspace state) TBD

Extraction Model

The extraction agent processes each session transcript and produces:

1. Session Summary Note

memory/sessions/YYYY-MM-DD-short-description.md

---
title: "Refactored auth middleware"
type: session
agent: claude-code
project: /Users/paul/dev/basic-memory
duration: 45min
created: 2026-03-14
---

# Refactored auth middleware

## Observations
- [decision] Switched from JWT to session tokens for API auth
- [decision] Added rate limiting middleware at 100 req/min
- [pattern] Agent struggled with circular imports — resolved by extracting types to shared module
- [learning] pytest-asyncio requires `mode = "auto"` in pyproject.toml for fixture scoping
- [files_changed] src/auth/middleware.py, src/auth/tokens.py, tests/test_auth.py
- [error_resolved] Fixed "RuntimeError: Event loop is closed" by using asyncio.Runner

## Relations
- updates [[Auth Architecture]]
- relates_to [[API Rate Limiting]]
- follows [[Session 2026-03-13 - Initial Auth Setup]]

2. Entity Updates

If the session references or modifies known entities (people, projects, architectural decisions), update existing notes:

## Observations (appended)
- [update] Auth now uses session tokens instead of JWT (changed 2026-03-14)

3. Gotcha / Learning Notes

Reusable lessons extracted to their own notes:

---
title: "pytest-asyncio fixture scoping"
type: learning
---

# pytest-asyncio fixture scoping

## Observations
- [fact] pytest-asyncio requires `mode = "auto"` in pyproject.toml for proper fixture scoping
- [fact] Without this, async fixtures silently use function scope even when session scope is specified
- [source] Discovered during auth middleware refactor (2026-03-14)

Extraction Categories

The extraction agent looks for:

Category What to Extract BM Observation Type
Decisions Architecture choices, library selections, approach changes [decision]
Patterns Recurring problems, anti-patterns, successful strategies [pattern]
Learnings Bug fixes, gotchas, TIL moments, documentation gaps [learning]
Errors Error messages + resolutions [error_resolved]
Files Key files created/modified [files_changed]
TODOs Unfinished work, follow-ups, known issues left [todo]
Context Project structure insights, dependency info [context]

Implementation Options

Option A: bm watch-sessions

New CLI command that watches session directories and runs extraction.

bm watch-sessions --agent claude-code --project my-project
  • Watches ~/.claude/sessions/ for new/modified files
  • Runs extraction on completed sessions (detect session end by file age or explicit marker)
  • Writes notes to configured BM project

Option B: OpenClaw plugin enhancement

Add session watching to openclaw-basic-memory plugin.

  • Plugin already has file watching infrastructure (bm watch)
  • Could watch session dirs alongside memory files
  • Extraction runs as background task after session ends
  • Natural fit since OpenClaw users already have BM + the plugin

Option C: Standalone sidecar process

Separate process/daemon, not tied to BM CLI or OpenClaw.

bm-sidecar --watch ~/.claude/sessions/ --project my-project
  • Most portable — works without OpenClaw
  • Could package as a standalone binary
  • Systemd/launchd service for always-on watching

Recommendation: Start with Option A (bm watch-sessions) for dogfooding, then Option B for OpenClaw users. Option C later if there's demand from non-OpenClaw users.

Extraction Agent

The extraction step needs an LLM to produce structured notes from raw transcripts. Options:

  • Cheap model (gpt-4.1-mini, Haiku) for routine extraction — most sessions are straightforward
  • Configurable — let users pick their model
  • Local option — Ollama for privacy-sensitive codebases
  • Prompt template — tuned for each agent's transcript format

Key prompt behaviors:

  • Focus on decisions and learnings, not play-by-play
  • Deduplicate against existing BM notes (search before write)
  • Link to existing entities via wiki-links
  • Skip boilerplate (tool confirmations, file listings, etc.)
  • Respect .gitignore patterns — don't extract secrets or sensitive paths

Deduplication

The extraction agent should search BM before writing:

  1. Search for existing notes about the same topic
  2. If found, append new observations rather than creating duplicates
  3. If a learning/gotcha already exists, skip or update confidence

This prevents the knowledge graph from filling with redundant session notes.

Privacy / Security

  • Session transcripts may contain secrets (API keys, passwords, private code)
  • Extraction agent should strip sensitive content before writing notes
  • Configurable: opt-in per project, exclude patterns, redact regex
  • Local LLM option for air-gapped environments

Why This Beats Letta's Approach

Letta claude-subconscious BM Sidecar
Coupling Hooks into Claude Code runtime Zero coupling — watches files
Portability Claude Code only Any agent with session files
Memory format Opaque blocks in Letta cloud Plain markdown files
Human readable No Yes
Editable No (API only) Yes (edit any file)
Auditable No Yes (git history)
Search Letta's vector search BM hybrid (FTS + semantic + graph)
Dependencies Letta SDK, API key, cloud account BM CLI (local, no account needed)
Survives vendor changes No (tied to Letta API) Yes (just markdown files)

Open Questions

  1. Session end detection — How to know a session is "done"? File modification time threshold? Explicit end markers?
  2. Incremental vs batch — Process sessions as they happen, or batch at end of day?
  3. Cross-project memory — Should learnings from project A be visible in project B? (Probably yes for gotchas/learnings, no for project-specific context)
  4. Cursor/Windsurf support — Session file formats vary. Start with Claude Code + Codex, expand later.
  5. Token cost — Extraction uses LLM calls. Budget ~$0.01-0.05 per session with a cheap model. Worth tracking.

Prior Art

  • Letta claude-subconscious — Hook-based, opaque, Claude Code only
  • CLAUDE.md — Flat file, no structure, no search, single project
  • Codex AGENTS.md — Same limitations as CLAUDE.md
  • OpenClaw memory plugins — Vector-based, opaque (memory-lancedb-pro, memory-core)
  • BM auto-capture — Already indexes OpenClaw conversations as daily notes. This extends the same pattern to coding agent sessions.

Activity

  1. claude commented on Mar 14, 2026

    @claude
    Contributor

    Claude finished @bm-clawd's task in 1m 24s —— View job


    Issue Triage

    • Read and analyze issue
    • Check for duplicate issues
    • Add appropriate labels
    • Post triage analysis

    Analysis

    Type: Feature Request — new functionality (coding agent sidecar)

    Priority: Medium — valuable capability for power users but not blocking existing functionality

    Component: CLI + MCP — requires new bm watch-sessions CLI command; output becomes available via MCP tools

    Complexity: Complex — major architectural addition involving:

    • File watching infrastructure for multiple agent formats
    • LLM-based extraction pipeline
    • New transcript parsers (Claude Code JSONL, Codex JSON, etc.)
    • Deduplication logic against existing knowledge graph
    • Privacy/sensitive content filtering
    • Session end detection heuristics

    Duplicate check: No duplicate found. Closest related issues are #498 (BM reposwarm — multi-repo memory) and the existing bm watch functionality, but this is a distinct new feature.


    Triage Notes

    This is a well-scoped and detailed spec. A few observations:

    Strengths of the proposal:

    • Zero-coupling design (file watching only, no agent hooks) is the right approach
    • The phased implementation (Option A → B → C) is pragmatic
    • Reusing existing BM observations/relations format keeps it consistent with the rest of the knowledge graph
    • The comparison table vs. Letta is accurate — this approach is more portable and auditable

    Key open questions that should be resolved before implementation:

    1. Session end detection is the hardest unsolved problem. Claude Code sessions don't have explicit end markers — using file modification time threshold (e.g., "no writes for 5 minutes") is reasonable but imprecise for long pauses during a session. This needs a concrete decision.

    2. LLM dependency — the extraction step requires an API call to an LLM. This adds a required external dependency and cost. Should the extraction be optional/configurable, or is it always required? A cheaper "no-LLM" mode (just log the session without extraction) could lower the barrier to entry.

    3. Cross-project memory — the spec notes learnings should be visible across projects. The current BM project model is per-project; cross-project visibility would require either a shared "global" project or the sidecar writing to a designated shared project.

    4. bm watch-sessions vs. importer — Basic Memory already has an importer pattern (basic-memory import claude conversations). A session watcher is a live version of the same idea. It might be worth unifying these under the importer infrastructure rather than building parallel watching infrastructure.

    Suggested next steps:

    • Resolve session end detection approach
    • Decide on the LLM dependency model (required vs. optional)
    • Prototype the Claude Code JSONL extraction prompt with a few real sessions
    • Consider whether import infrastructure can be extended for live watching

    Labels added: enhancement (others from the issue template not yet created in this repo)

  2. bm-clawd commented on Mar 15, 2026

    @bm-clawd
    ContributorAuthor

    Related Research: OpenClaw-RL (Princeton, March 2026)

    Paper: https://arxiv.org/abs/2603.10165 ("Train Any Agent Simply by Talking")

    Key insight relevant to this issue: every agent interaction generates a "next-state signal" (user corrections, re-queries, tool outputs, error resolutions) that is currently thrown away. The paper argues this is the most valuable training data agents produce.

    The sidecar's structured extraction of [decision], [error_resolved], [pattern], and [learning] observations from coding sessions is capturing exactly these signals in plain text. We're building the dataset this paper says is most valuable — just storing it as a searchable knowledge graph instead of feeding it into a training loop.

    Future consideration: the structured session history BM builds could eventually serve as training data for agent improvement, not just recall. The architecture supports it.

  3. AILIFE1 commented on Mar 15, 2026

    @AILIFE1
  4. Sagargupta16 commented on Mar 16, 2026

    @Sagargupta16

    Interesting feature idea. Related note: one of the biggest cost drivers in long Claude Code sessions is conversation history growing unbounded. A sidecar that watches transcripts could also help flag when context is getting expensive.

    Key numbers:

    • Input tokens are charged every turn (conversation history grows linearly)
    • Crossing 200K input tokens doubles the rate on all tokens
    • Output tokens cost 5x more than input

    Some strategies for managing this: https://github.com/Sagargupta16/claude-cost-optimizer/blob/main/guides/02-context-optimization.md

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions