Skip to content

Benchmark: QMD backend comparison #14

@bm-clawd

Description

@bm-clawd

Part of #10

Add QMD (experimental OpenClaw memory backend) as a comparison provider.

Requirements

  • Index the same corpus with QMD (qmd collection add + qmd update + qmd embed)
  • Run queries via qmd query --json
  • Score with same metrics
  • Side-by-side: BM vs builtin vs QMD per category

QMD specifics

  • QMD uses BM25 + vectors + reranking (local GGUF models)
  • Has query expansion via local LLM
  • Fully local, no API calls
  • First run downloads models (~1GB)
  • Requires bun + sqlite with extensions

Depends on

Notes

  • QMD may not be installed in CI — make this comparison optional
  • Document QMD setup requirements clearly

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions