Skip to content

feat(memory): import Claude Code auto-memory into AutoBot knowledge base #16642

Description

@mrveiss

Context

AutoBot already has a memory architecture (autobot-backend/memory/): EssentialStoryGenerator (compact always-loaded summary), WorkingMemoryService (Redis, session-scoped, TTL), GeneralStorage/TaskStorage, and knowledge_facts (Postgres system-of-record, Redis+ChromaDB projections, declared in autobot_shared/store_authority.py).

Separately, Claude Code sessions on dev machines maintain an auto-memory store at ~/.claude/projects/<project-slug>/memory/*.md — each file has frontmatter (name, description, metadata.type: user|feedback|project|reference) plus a body. This is curated project knowledge (decisions, gotchas, process feedback) accumulated across many coding sessions, currently invisible to AutoBot's own agents/chat.

Goal

A reusable, idempotent importer that reads Claude Code auto-memory files and writes them into AutoBot's existing knowledge_facts store (via the existing knowledge.facts / FactsMixin write path — no new store, no bypass of the declared system-of-record), so AutoBot's own KB/RAG can surface this knowledge.

Location

  • Write path to reuse: autobot-backend/knowledge/facts.py (FactsMixin)
  • Store declaration: autobot_shared/store_authority.py (knowledge_facts concept)
  • Admin-gated pattern to follow: autobot-backend/api/knowledge_population.py (check_admin_permission)

Acceptance Criteria

  • Importer (CLI script or admin endpoint) parses <memory-dir>/*.md frontmatter + body; memory directory path is configurable (SSOT/env), never hardcoded
  • Each memory file becomes one knowledge_facts entry tagged with its metadata.type and a source marker (e.g. category="claude_code_memory")
  • Writes go exclusively through the existing knowledge.facts write path declared in store_authority.py for knowledge_facts
  • Idempotent: re-running updates existing facts (keyed by memory file slug) rather than duplicating
  • Gated by admin permission (reuse check_admin_permission, consistent with knowledge_population.py)
  • Unit tests: frontmatter parsing, idempotent upsert behavior
  • Evidence: fact count before/after against a running instance, plus one imported fact retrievable via KB search

Out of Scope

  • Raw Claude Code session transcripts (JSONL) — a separate, much larger data source
  • AutoBot's own chat_workflow session history — unrelated data source

Activity

  1. mrveiss commented on Sep 14, 2026

    @mrveiss
    OwnerAuthor

    Progress update — implementation is in PR #16643, currently parked.

    Status: blocked, not a stall. Three security review rounds found real issues (all fixed except one pre-existing gap independent of this feature) plus two CI-mechanics violations (also fixed). Current head: 4d677c399.

    Acceptance criteria status:

    • Importer parses <memory-dir>/*.md frontmatter + body, configurable via AUTOBOT_CLAUDE_MEMORY_DIR env (never hardcoded) — knowledge/claude_memory_importer.py
    • Each memory file → one knowledge_facts entry tagged category="claude_code_memory" + metadata.type
    • Writes go exclusively through the existing knowledge.facts write path (store_fact/update_fact) — no bypass of store_authority.py's declared system-of-record
    • Idempotent — deterministic fact_id_for(slug), re-run updates rather than duplicates
    • Admin-gated (check_admin_permission, router-level) — api/knowledge_claude_memory.py
    • Unit tests — 25 tests: frontmatter parsing, idempotent upsert, redaction (7 categories), owner/visibility metadata + enforcement proof
    • Not yet met: "fact count before/after against a running instance, plus one imported fact retrievable via KB search" — requires a live host run, which requires merging first

    Blocked on #16654 (canonical KB search + grounded-agent RAG enforce no ownership/visibility at all — pre-existing gap, independent of this feature, found during review; every PRIVATE-scoped fact in the system is exposed the same way today). #16507 was the other blocker; already resolved (merged via #16508).

    Non-blocking follow-ups filed during review, not required for this issue's own ACs: #16655 (exact-match fleet-hostname/IP redaction, closes a disclosed residual gap), #16656 (isolated knowledge buckets — better long-term home, not required now), #16661 (folder-watch continuous auto-import, an enhancement on top of the on-demand endpoint this issue asked for).

    Full detail, evidence, and the review thread: PR #16643.

  2. added this to the v0.11.0 milestone on Sep 14, 2026
  3. mrveiss commented on Oct 3, 2026

    @mrveiss
    OwnerAuthor

    Closure check against origin/main (1edec3064c): 6 of 7. The last one is host evidence, and it blocks #16661

    Criterion Evidence
    Importer parses <memory-dir>/*.md frontmatter + body; path configurable, never hardcoded autobot-backend/knowledge/claude_memory_importer.py: parse_memory_file (:96), iter_memory_files (:123, excludes MEMORY.md). get_claude_memory_dir (:45-58) reads AUTOBOT_CLAUDE_MEMORY_DIR, else derives the path from project_root() and Path.home() with no machine path baked in. The comment at :38-41 records why it is not in ssot_config (frozen size ceiling).
    One knowledge_facts entry per file, tagged with metadata.type and a source marker CATEGORY = "claude_code_memory" (:36); _fact_metadata sets category and memory_type (:209-212)
    Writes only through the declared knowledge.facts path module docstring (:11-13): upserts via store_fact / update_fact, the path store_authority.py declares
    Idempotent, keyed by slug fact_id_for(slug, owner_id) (:138); test_import_memory_file_updates_when_fact_present, test_import_claude_memory_is_idempotent_on_rerun (claude_memory_importer_test.py:263, :386)
    Admin-gated api/knowledge_claude_memory.py:35 dependencies=[Depends(check_admin_permission)]; there is no caller-supplied path (:49-50)
    Unit tests: frontmatter parsing, idempotent upsert claude_memory_importer_test.py: test_parse_memory_file_* (:149-183), the idempotency tests above, plus owner scoping and redaction (:228-535)
    Evidence: fact count before/after on a running instance, and one imported fact retrievable via KB search Not met. No such evidence is recorded in this issue or its PRs. It needs an admin-authenticated POST /import_claude_memory against a deployed instance, then a KB search. I did not run it; it requires admin credentials, which I do not read.

    Why this one matters more than its tick count suggests: this issue blocks #16661 (continuous auto-import), so #16661 cannot be picked while one host observation is outstanding. Whoever can run an admin call against the deployed instance can close both the observation and the edge.

  4. added
    blocked: needs-observationNothing but an observation on a running system is left; no session can close it
    on Oct 3, 2026
  5. mrveiss commented on Oct 3, 2026

    @mrveiss
    OwnerAuthor

    What discharges the evidence criterion, and two things that would make the obvious run wrong

    On the live instance, the importer's default directory does not exist for the backend service user, and AUTOBOT_CLAUDE_MEMORY_DIR is not in the unit's Environment= (EnvironmentFile contents not read). get_claude_memory_dir() falls back to ~/.claude/projects/<project_root-with-/-as-->/memory (claude_memory_importer.py:45-58), which for a deployed checkout is a path nothing creates. A bare POST /import_claude_memory should therefore import zero facts, and zero would read as a failure of the importer rather than a missing directory.

    The second thing is #16654. It is still open: fact visibility is not enforced on the canonical KB search paths. Pointing the import at the real memory directory would make the owner's whole memory directory (hundreds of files) searchable by any user until that lands. Redaction covers 7 detector categories, not internal context.

    Cheapest discharge that avoids both:

    1. Create a directory readable by the service user holding one synthetic *.md memory file with frontmatter, and set AUTOBOT_CLAUDE_MEMORY_DIR to it for the backend service (a config change to the live service, so through the builtin path, not a shell edit).
    2. As an admin, record the knowledge_facts count, POST /import_claude_memory, poll its status, record the count again.
    3. Search the KB for a phrase from the synthetic file and confirm the fact is returned with category="claude_code_memory".

    That discharges the one open criterion and the blocks edge into #16661 (native edge confirmed: #16661 is blocked_by #16642).

    Batches with #16280: one admin session.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions