Repository navigation
fix(grounded): case-insensitive claim annotation + verify causal CRITICAL path (#11248) - #11285
Merged
Merged
Conversation
…ICAL path (#11248) Two #11248 items, biggest-gain fixes: grounded_agent._reconstruct_response used case-SENSITIVE 'claim_text in reconstructed', so LLM-extracted claims whose case differs from the source text (the common case) silently got no KB-source annotation. Match case-insensitively via re.search(IGNORECASE) while preserving the response's original casing in the output (don't rewrite the user's text). Real robustness fix; test updated to the new (correct) output. Verified on py3.10 and py3.12. test_causal_inference_engine::test_critical_severity expected CRITICAL but the code intentionally downgrades CRITICAL->DEGRADED when a high-confidence fix exists (documented behaviour). Lower the test's intervention below that threshold so it verifies the true CRITICAL classification path with zero behaviour change. No regressions (test_grounding_mixed + test_resolve_conflict already fail on base — separate pre-existing #11248-class issues). black/isort/compile clean.
Contributor
✅ SSOT Configuration Compliance: Passing🎉 No hardcoded values detected that have SSOT config equivalents! |
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Thinking Path
The two #11248 items you asked me to decide (biggest-gain wins):
grounded_agent (real robustness bug) —
_reconstruct_responsematched claimscase-sensitively (
claim_text in reconstructed), so an LLM-extracted claimwhose case differs from the source (e.g. claim
"Latency increased"vs response"the latency increased") silently got no KB-source annotation. Biggest gain:match case-insensitively and preserve the response's original casing in the
output (don't rewrite the user's text). Fixes the production bug; source text stays
intact.
causal severity (zero-risk path fix) —
test_critical_severityexpected CRITICAL,but
_assess_severityintentionally downgrades CRITICAL→DEGRADED when ahigh-confidence fix exists (documented in code). Rather than change that documented
semantics (a separate product call), make the test verify the true CRITICAL path
with a sub-threshold intervention — zero behaviour change.
What Changed
services/grounded_agent.py— case-insensitivere.search(..., IGNORECASE),preserve matched (response) casing; annotate first match. (
import readded.)tests/services/test_grounded_agent.py— assert the annotation fired + claimpresent case-insensitively (was asserting the claim's exact capitalisation).
tests/services/test_causal_inference_engine.py— intervention 0.9/0.9 → 0.6/0.6so the documented high-confidence-fix downgrade doesn't fire.
Verification
test_reconstruct_response_with_annotation: passes on py3.10 and py3.12 (was failing both).test_critical_severity: passes.test_grounding_mixed+test_resolve_conflictalready fail onbase (separate pre-existing test-remediation: 10 tests/services files fail scoped (8 real assertion failures + 2 conftest-stub artifacts) #11248 items: Redis-None + verification-returns-0).
py_compile+black --check -l120+isortclean.Model Used
Opus 4.8 (1M context)
Refs #11248.