perf: avoid full-width post-clean narrative profiling - #181
Conversation
|
Important Review available on request
Reviews should be triggered manually for repositories with fewer than 10 stars. Select Trigger review above or comment ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Strix is installed on this repository, but we couldn't run this PR security review because this workspace's trial has ended. Add a card to resume code reviews here. So far, Strix has reviewed 36 pull requests, surfaced 3 security issues (1 critical/high) and blocked 1 risky merge across this workspace. |
Summary
explain_clean()'s post-clean context build to columns that can actually contribute a narrativeWhy
explain_clean()unconditionally profiled every cleaned column even though_narratives()only reads contexts for columns with rationale-bearing actions or for columns that remain missing. On a wide no-op frame, that doubled per-column context construction from 120 to 240 calls. Reusing the pipeline cache directly would be unsafe because it is built after representation repair and only covers engine-visible columns, while the role table describes the original frame.The new selection preserves the existing narrative semantics while avoiding the redundant full-width pass.
Closes #32
Validation
pytest -m "not online and not large"— 4,052 passed, 6 skipped, 12 deselected; 93.16% coveragepytest --no-cov -q tests/test_explain.py— 14 passedruff check src testsmypy src/freshdata— no issues in 201 source filesgit diff --checkAI assistance
I used OpenAI Codex to inspect the current profiling flow, implement the optimization, and run validation. I reviewed the final diff and test evidence before submission.