Skip to content

[P0] Helpfulness-driven rerank weight tuning v1 #5

@justinbuzzni

Description

@justinbuzzni

Goal

  • Improve retrieval quality over time using user/session helpfulness signals.

Scope

  • Collect helpful/neutral/unhelpful feedback signals.
  • Periodically tune lexical/semantic/recency weights.
  • Add rollback guard when quality drops.

Acceptance Criteria

  • Top1 +5%p OR Top3 +8%p improvement on replay benchmark.
  • Automatic rollback path exists.

Checklist

  • Define feedback schema
  • Implement tuner job (v1)
  • Add rollback guardrail
  • Produce before/after quality report

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions