Skip to content

feat(eval): Chat Eval Pipeline Phase 1-4 + fix#548

Merged
mangowhoiscloud merged 2 commits intodevelopfrom
feat/chat-eval-pipeline
Feb 9, 2026
Merged

feat(eval): Chat Eval Pipeline Phase 1-4 + fix#548
mangowhoiscloud merged 2 commits intodevelopfrom
feat/chat-eval-pipeline

Conversation

@mangowhoiscloud
Copy link
Contributor

Summary

  • Phase 1: Domain + Application layer (EvalResult VO, EvalConfig DTO, L1/L2/L3 서비스)
  • Phase 2: BARS prompts, LangGraph eval subgraph (Send API fan-out), main graph 통합
  • Phase 3: Gateway adapters (Redis + PG composite), DI wiring, V005 migration, calibration fixture
  • Phase 4: PG pool DI (asyncpg), SSE eval stage, structured logging, eval-feedback-loop skill
  • Fix: 누락된 persistence 어댑터, calibration fixture, 단위 테스트, factory wiring 추가

Key Numbers

  • 36 files across Domain/Application/Infrastructure layers
  • 165 tests ALL PASS (pytest -m eval_unit)
  • 5-expert review loop: Design R5(99.8) / Code R2(97.1)

Architecture

  • Swiss Cheese 3-Tier: L1 Code Grader → L2 LLM BARS Grader → L3 Calibration CUSUM
  • Clean Architecture: Port/Adapter, CQRS (Command/Query Gateway), frozen VO
  • FAIL_OPEN: eval 실패 시 B-grade(65.0) fallback, 사용자 응답 차단 없음

Test plan

  • pytest -m eval_unit -v --tb=short — 165 tests ALL PASS
  • black --check && ruff check — CI lint clean
  • Redis-only 모드 동작 확인 (eval_postgres_dsn 비어있을 때)
  • SSE 이벤트에 eval 단계 표시 확인
  • enable_eval_pipeline=True 기본 동작 확인

🤖 Generated with Claude Code

mangowhoiscloud and others added 2 commits February 10, 2026 03:30
Phase 3에서 생성된 infrastructure/persistence/eval/ 디렉토리가
커밋에 누락되어 ModuleNotFoundError 발생. 추가.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- calibration_monitor: recalibrate() stub for HITL
- factory.py: eval_counter param passthrough to create_chat_graph
- calibration_set.json: fixture data for L3 calibration
- V005 migration: eval schema DDL
- 4 adapter unit tests (composite gateway, json calibration, redis counter, redis adapter)
- integration test __init__.py

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@mangowhoiscloud mangowhoiscloud self-assigned this Feb 9, 2026
@github-project-automation github-project-automation bot moved this to Backlog in @Eco² Feb 9, 2026
@mangowhoiscloud mangowhoiscloud merged commit e92ccf8 into develop Feb 9, 2026
15 checks passed
@github-project-automation github-project-automation bot moved this from Backlog to Done in @Eco² Feb 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant