You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Discovery during PR #7131 (#7041) — pre-existing benchmark file breakage
PR #7131 migrated performance_benchmarks.performance_test.py off the deleted LLMInterface and onto LLMService. While doing so, three pre-existing breakage sites surfaced that predate #3185 — the file has not been runnable in CI for a long time. Filing for tracked cleanup.
The repo only has enhanced_memory_manager_async.py — the synchronous module was removed at some point (likely the async migration). The import fails at module load with ModuleNotFoundError.
2. knowledge_base import chain broken (line 20)
fromknowledge_baseimportKnowledgeBase
knowledge_base.py exists, but importing it raises ModuleNotFoundError: No module named 'models.task_context' — a transitive dep that's missing.
3. Non-standard filename → not pytest-collected
performance_benchmarks.performance_test.py doesn't match pytest's default test_*.py / *_test.py glob. Even if (1) and (2) were fixed, the file wouldn't run on CI without explicit --collect-only or a config addition.
Acceptance criteria
Decide between:
A. Resurrect — fix the 3 gaps:
Migrate EnhancedMemoryManager import to enhanced_memory_manager_async (or its successor); update the test class accordingly
Fix or stub models.task_context so knowledge_base.py imports cleanly in test contexts
Either rename the file to performance_benchmarks_test.py (pytest-collected by default) OR add an explicit pytest.ini rule for the .performance_test.py extension AND mark with @pytest.mark.benchmark for opt-in
Verify benchmarks actually run with a real Ollama instance and produce meaningful numbers
B. Retire — delete the file after confirming with the project maintainer that benchmark coverage is reproducible elsewhere (or filed as a separate "build new benchmark suite" issue). Per project policy [Never delete code — wire it in (ABSOLUTE)] this is the less preferred path.
The runtime-correctness layer of (A) is significant — at minimum it requires:
Ollama running with llama3.2:1b and nomic-embed-text available
Knowledge base initialized
Memory manager initialized
That's effectively a "benchmark e2e suite" effort, separate from PR #7131's import cleanup.
Activity
fix(tests): restore performance_benchmarks tests — fix imports + rena…
mrveiss commentedon May 16, 2026
✅ Closed with proof of implementation
Commit: 2b86e73 fix(tests): restore performance_benchmarks tests — fix imports + rename for pytest collection
Acceptance Criteria Met:
Evidence: Merged to Dev_new_gui