[Feature]: Background Memory Manager for Automatic Extraction/Consolidation

[zircote]

Problem Statement

Memory maintenance tasks (consolidation, entity extraction, freshness updates) currently require explicit invocation. This means:

• Users must remember to run maintenance
• Optimal timing is unclear
• System health depends on manual intervention

LangMem implements a background memory manager that handles extraction and consolidation automatically based on configurable triggers.

Proposed Solution

Implement a background memory manager daemon:

Automatic Operations:

• Entity extraction on new memories
• Periodic consolidation based on memory growth
• Freshness updates based on access patterns
• Index optimization during low usage

Triggers:

• Time-based (cron-like scheduling)
• Event-based (after N new memories)
• Threshold-based (when similarity clusters detected)
• Resource-based (when system idle)

Proposed API:

pub struct BackgroundManager {
    config: BackgroundConfig,
    scheduler: Scheduler,
    workers: Vec<Worker>,
}

pub struct BackgroundConfig {
    consolidation: ConsolidationTrigger,
    entity_extraction: ExtractionTrigger,
    freshness: FreshnessTrigger,
    enabled: bool,
}

pub enum Trigger {
    Cron(String),
    MemoryCount(usize),
    Threshold { metric: Metric, value: f32 },
    Idle { duration: Duration },
}

Configuration in config.toml:

[background]
enabled = true

[background.consolidation]
trigger = "memory_count"
threshold = 100
namespaces = ["decisions", "learnings"]

[background.entity_extraction]
trigger = "immediate"  # Extract on capture

CLI:

subcog background status
subcog background run consolidation
subcog background enable/disable

Alternatives Considered

• Manual invocation only (current) - poor UX
• External cron jobs - fragile, loses context
• Always-on processing - resource wasteful

Additional Context

• LangMem background manager: https://langchain-ai.github.io/langmem/
• Extends existing consolidation with automation
• Could be optional daemon process

Breaking Change: No
Priority: Nice to have
Contribution: Yes, with guidance