Summary
A semantic cache reuses a stored answer when a new query scores above similarity_threshold (default 0.8) against a cached query. The threshold assumes: high similarity ⇒ same question. For the query pairs a cache must not merge a question and its reversal, as the assumption runs backwards. Further, reversing a query is a minimal edit that keeps almost all tokens, so it scores far above 0.8, while an equivalent query phrased freshly can score below it. The result is the worst trade for a cache: false hits on opposite-meaning queries, misses on same-meaning ones. The similarity-evaluation tests cover paraphrase-like positives; none of them pit shared wording against opposite meaning. This is a test blind spot report, not a claim the cache is broken (no cosine cut separates these classes) which is why the tests should pin the boundary.
Minimal reproducible case
Real specimen pairs from a released audit corpus, cosines measured under nomic-embed-text-v1.5 (256-dim MRL truncation; frozen per-pair results ship with the paper's artifact, and values shift by embedding model, but bottom line is this mutation class scored 0.83–0.9997 across all nine encoder configurations we tested):
- Cached query: "Withhold the study drug from any participant who reports chest tightness."
New query: "Administer the study drug to any participant who reports chest tightness."
Cosine 0.9608 > 0.8 → cache hit. The user asking about administering receives the cached answer about withholding.
- Cached query: "Limit each household to at most 3 emergency food boxes per week."
New query: "Limit each household to at most 30 emergency food boxes per week."
Cosine 0.9980 > 0.8 → cache hit across a 10× quantity change.
The mirror case: "Retry the request at most three times." vs. the equivalent "Give the call up to three attempts, then stop." averages cosine ≈ 0.76 under the same encoder → cache miss on a reusable answer.
from gptcache import Config
# default: Config().similarity_threshold == 0.8
# With any embedding-cosine SimilarityEvaluation, the withhold/administer pair
# above evaluates above threshold and returns the cached completion.
Proposal
Add polarity cases to the similarity-evaluation test suite, per §11.3 of the audit paper (published: https://arxiv.org/abs/2608.10216):
- Query pairs sharing wording, with opposite decisions (negation flip,
must→may, quantity/unit changes, scope inversion) asserted as must-NOT-hit fixtures at the default threshold, xfail where they currently hit.
- Query pairs sharing the decision with no shared wording are documented as the miss-side boundary.
For safety adjacent domains (the clinical pair above is real), a recommendation therein is to disable semantic caching, or require exact match, would follow directly from the fixtures. Happy to contribute the cases as a PR; the corpus is released with the paper's artifact.
Summary
A semantic cache reuses a stored answer when a new query scores above
similarity_threshold(default 0.8) against a cached query. The threshold assumes: high similarity ⇒ same question. For the query pairs a cache must not merge a question and its reversal, as the assumption runs backwards. Further, reversing a query is a minimal edit that keeps almost all tokens, so it scores far above 0.8, while an equivalent query phrased freshly can score below it. The result is the worst trade for a cache: false hits on opposite-meaning queries, misses on same-meaning ones. The similarity-evaluation tests cover paraphrase-like positives; none of them pit shared wording against opposite meaning. This is a test blind spot report, not a claim the cache is broken (no cosine cut separates these classes) which is why the tests should pin the boundary.Minimal reproducible case
Real specimen pairs from a released audit corpus, cosines measured under
nomic-embed-text-v1.5(256-dim MRL truncation; frozen per-pair results ship with the paper's artifact, and values shift by embedding model, but bottom line is this mutation class scored 0.83–0.9997 across all nine encoder configurations we tested):New query: "Administer the study drug to any participant who reports chest tightness."
Cosine 0.9608 > 0.8 → cache hit. The user asking about administering receives the cached answer about withholding.
New query: "Limit each household to at most 30 emergency food boxes per week."
Cosine 0.9980 > 0.8 → cache hit across a 10× quantity change.
The mirror case: "Retry the request at most three times." vs. the equivalent "Give the call up to three attempts, then stop." averages cosine ≈ 0.76 under the same encoder → cache miss on a reusable answer.
Proposal
Add polarity cases to the similarity-evaluation test suite, per §11.3 of the audit paper (published: https://arxiv.org/abs/2608.10216):
must→may, quantity/unit changes, scope inversion) asserted as must-NOT-hit fixtures at the default threshold, xfail where they currently hit.For safety adjacent domains (the clinical pair above is real), a recommendation therein is to disable semantic caching, or require exact match, would follow directly from the fixtures. Happy to contribute the cases as a PR; the corpus is released with the paper's artifact.