feat(stats): introduce privacy-first gains tracking feature @hansipie

[hansipie]

Summary

This PR introduces a privacy-first gains monitoring system for grepai. Every search and trace command now automatically records token usage metrics locally, and a new grepai stats command lets users visualize how much they save compared to a traditional grep-based workflow.

---

Motivation

Users had no visibility into the concrete value grepai brings over grep. This feature answers the question: "How many tokens (and dollars) have I actually saved by using grepai?"

---

What's new

New stats/ package

• Entry — one record per search/trace command (timestamp, command type, output mode, result count, tokens consumed, grep-equivalent tokens)
• Recorder — appends entries as NDJSON to .grepai/stats.json using a file lock (cross-platform: flock on Unix, LockFileEx on Windows)
• ReadAll / Summarize / HistoryByDay — aggregation helpers
• GrepEquivalentTokens(n) — estimates what grep would have cost: n × 512 tokens × 3 (expansion factor for context)
• Cost estimation for cloud providers (OpenAI, OpenRouter) at $5/M tokens; nil for local providers (Ollama, LM Studio)
• 13 unit tests with -race, all green

New grepai stats command (cli/stats.go)

grepai stats [--json] [--history] [--limit N]

• Human-readable output with lipgloss (rounded border, colors)
• --json — structured JSON output for scripting/agents
• --history — per-day breakdown table
• --limit N — max days shown (default: 30)

Auto-recording in search and trace

Stats are recorded fire-and-forget in a goroutine with a 100 ms timeout — zero impact on search/trace latency. Output is captured to a string (instead of writing directly to stdout) to enable token counting without stdout hijacking.

MCP integration (mcp/server.go)

Stats are recorded for all MCP tool calls: grepai_search, grepai_trace_callers, grepai_trace_callees, grepai_trace_graph. A new grepai_stats MCP tool allows AI agents to query savings programmatically.

---

Example output

╭─────────────────────────────────────────────────────╮
│ grepai stats — Token Savings Report                 │
│                                                     │
│ Total queries      42                               │
│ Tokens (grepai)    8,320                            │
│ Tokens (grep est.) 64,512                           │
│ Tokens saved       56,192  ▲ 87.1%                  │
│ Est. cost saved    $0.2810  (cloud provider)        │
│                                                     │
│ By command:  search 38 · trace-callers 4            │
│ By mode:     compact 30 · full 12                   │
╰─────────────────────────────────────────────────────╯

Testing

make test        # all packages pass with -race
make build       # binary builds clean
grepai stats --help
grepai stats --json
grepai stats --history --limit 7

Type of Change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation update
• Refactoring (no functional changes)
• Performance improvement
• Test update

How Has This Been Tested?

• Unit tests
• Integration tests
• Manual testing

Test Configuration:

• OS: Linux Windows
• Go version:
• Embedding provider:

Checklist

• My code follows the project's code style
• I have run golangci-lint run and fixed any issues
• I have added tests that prove my fix/feature works
• I have updated the documentation if needed
• I have added an entry to CHANGELOG.md (if applicable)
• My changes generate no new warnings
• All new and existing tests pass

Screenshots (if applicable)

Additional Notes

config_test:
On Windows, creating symlinks requires elevated privileges or Developer
Mode. Skip the test gracefully on Windows; keep t.Fatalf on other platforms.

[codecov]

Codecov Report

❌ Patch coverage is 25.57692% with 387 lines in your changes missing coverage. Please review.
✅ Project coverage is 44.27%. Comparing base (a322537) to head (67e9389).
⚠️ Report is 81 commits behind head on main.

Files with missing lines	Patch %	Lines
cli/trace.go	4.31%	111 Missing ⚠️
cli/stats.go	4.34%	110 Missing ⚠️
cli/search.go	0.00%	95 Missing ⚠️
mcp/server.go	30.76%	52 Missing and 2 partials ⚠️
stats/recorder.go	64.28%	6 Missing and 4 partials ⚠️
stats/reader.go	90.27%	4 Missing and 3 partials ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##             main     #162       +/-   ##
===========================================
+ Coverage   27.16%   44.27%   +17.11%     
===========================================
  Files          32       61       +29     
  Lines        3711    10335     +6624     
===========================================
+ Hits         1008     4576     +3568     
- Misses       2620     5361     +2741     
- Partials       83      398      +315     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[tinker495]

Thanks for the PR — I’m not a maintainer, but as a contributor/user I’m excited to see this merged.

Quick question/idea (feel free to ignore if it’s out of scope): since main recently introduced a Bubble Tea TUI for commands like status (#143, with a --no-ui fallback), would it make sense for grepai stats to follow the same pattern (or even surface stats inside the existing status UI) while keeping --json for scripting?

Totally fine if you’d rather keep this PR focused and tackle any TUI alignment as a follow-up.