Repository navigation
Combining-character (NFD) text never matches its precomposed (NFC) equivalent #70
Description
What happens
Fuzzy.Contains/CalculateScore (FuzzySearch/Fuzzy.cs, lines 46-69 and 99-200) compare subject and pattern purely per UTF-16 char, with no Unicode normalization. A precomposed character (e.g. é, U+00E9) and its canonically-equivalent decomposed form (e + combining acute accent, U+0301) are visually and semantically identical text, but are different code-unit sequences, so they're treated as a non-match.
Reproduction
Fuzzy.Contains("café", "café"); // → true (precomposed vs precomposed)
Fuzzy.Contains("café", "café"); // → false (decomposed vs precomposed — same visible text!)Why it matters
Decomposed (NFD) text is common in real input — e.g. filenames read from HFS+/APFS-backed filesystems, or text pasted from certain sources, are frequently NFD-normalized. Any consumer using FuzzySearch to match against such filenames or text will silently miss matches for accented/composed characters despite the visible text being identical to the search term.
Suggested fix
Normalize both spans to a common form (e.g. string.Normalize(NormalizationForm.FormC)) before comparing, or explicitly document this as a known limitation if avoiding the extra allocation on the hot path is a priority.
Acceptance criteria
Fuzzy.Contains(and the scoring overload) treat canonically-equivalent NFC/NFD forms of the same text as matching.- Add a test covering an NFD-decomposed subject or pattern against its NFC-precomposed counterpart.
Activity
matt-edmondson commentedon Sep 14, 2026
Triage
Category: Bug
Priority: Medium
Suggested assignment: none — no specific team/area indicated; owner is the FuzzySearch matching core (
FuzzySearch/Fuzzy.cs)Possible duplicate: none found — closely related to #71 (lone surrogate matching inside an emoji). Both are Unicode-correctness gaps in the same char-by-char matching loop in
Fuzzy.cs(lines 46-69, 99-200), and both are fixed by moving that loop off raw UTF-16charcomparison. Not duplicates — different defects, different acceptance criteria — but they should be scheduled and reviewed together, since fixing either one in isolation means touching the same hot path twice.Already in progress: no — no open PR in the org touches this
Why Medium: silent missed matches on input a consumer has no reason to suspect is different — NFD is the normal form for filenames on HFS+/APFS, which is a plausible primary use case for a fuzzy matcher. It stays below High because it is confined to accented/composed text and because "document as a known limitation" is a legitimate resolution here, given the allocation cost of normalizing on the hot path.
Suggested next step: this is the higher-value of the two Unicode issues and should be decided first, because the decision (normalize on the hot path vs. document the limitation) determines whether #71 is fixed as a side effect of codepoint-aware iteration or needs its own handling. If normalization is rejected on allocation grounds, measure before settling — a fast path that skips normalization when both inputs are already ASCII would avoid the cost for the common case.
Generated by Claude Code