fix(server): reject empty string list evaluator values#121
fix(server): reject empty string list evaluator values#121
Conversation
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
| @@ -24,3 +24,11 @@ class ListEvaluatorConfig(EvaluatorConfig): | |||
| description="'exact' for full string match, 'contains' for keyword/substring match", | |||
| ) | |||
| case_sensitive: bool = Field(False, description="Whether matching is case sensitive") | |||
There was a problem hiding this comment.
This catches exact "" but allows whitespace-only strings like " " or "\t" through. Those produce similarly pathological regexes - e.g. \b(\ )\b in contains mode.
Consider using strip():
if any(isinstance(value, str) and value.strip() == "" for value in values):
raise ValueError("values must not contain empty or whitespace-only strings")Same change needed in the defensive filter in evaluator.py:38 (if value != "" -> if value.strip()).
There was a problem hiding this comment.
Addressed in 319eede.
I broadened the guard from exact "" to any blank/whitespace-only string in both the config validator and the evaluator fallback. That now rejects values like " " and "\t" at validation time and defensively drops them for legacy invalid configs.
|
|
||
| # Then: the evaluator ignores the empty control values | ||
| assert result.matched is False | ||
| assert result.message == "Empty control values - control ignored" |
There was a problem hiding this comment.
This tests the denylist path (match_on="match"), but the original report was about an allowlist blocking all calls. Could we add a companion case with match_on="no_match" that asserts matched is False for normal input? That encodes the failure mode that prompted this fix.
Something like:
async def test_legacy_empty_string_allowlist_does_not_block_all(self) -> None:
config = ListEvaluatorConfig.model_construct(
values=[""],
logic="any",
match_on="no_match",
match_mode="exact",
case_sensitive=False,
)
evaluator = ListEvaluator(config)
result = await evaluator.evaluate("legitimate_value")
assert result.matched is FalseThere was a problem hiding this comment.
Addressed in 319eede.
I added a companion legacy allowlist regression covering match_on="no_match", plus a whitespace-only legacy case. The evaluator tests now cover both the original failure mode and the broadened blank-string hardening.
Summary
Testing
Fixes #120