Skip to content

A namesake tie goes to review, not to candidate order - #329

Merged
WaylandYang merged 3 commits into
deeplethe:devfrom
ousamabenyounes:fix/issue-270
Sep 4, 2026
Merged

WaylandYang merged 3 commits into
deeplethe:devfrom
ousamabenyounes:fix/issue-270

Conversation

@ousamabenyounes

Copy link
Copy Markdown
Contributor

Fixes #270.

The tie

Once a base holds two entities with the same name — Zhang Wei A (Platform Engineering) and B (Finance) — a later document that mentions Zhang Wei with no handle is attributed by profile similarity. When both were seeded from the same chunk their profile vectors are identical, so the new mention scores the same against both. resolve_mention kept the first candidate it met (sim > s, strict) and, once that score cleared SIM_ATTACH, attached silently. No review pair was filed. Which entity got the fact depended on the order the candidates came back from the database.

The fix

A near-tie among namesakes is now a grey zone regardless of the absolute score (per the issue's first option). When the top-scored candidate would otherwise attach (sim >= SIM_ATTACH) but another candidate with the same canonical name scores within a new SIM_TIE_MARGIN of it, the mention attaches to neither: it creates the entity and files a review pair against each namesake at the human stage.

Filing at human (not adjudicating) is the point — the batch adjudicator (adjudication.rs) auto-merges adjudicating pairs, which is exactly what must not happen to two profiles it cannot tell apart. pending_adjudications filters stage = 'adjudicating', so human pairs never reach it, and create_review's upsert only ever escalates a pre-existing pair to human.

  • SIM_ATTACH / SIM_NEW are untouched, as the issue asked.
  • Name comparison in the tie check is case-insensitive, matching recall's SQL lower().
  • To let resolve_mention carry the review stage out, ReviewRequest gained a stage field; every existing caller keeps Adjudicating, and the extraction loop now honours it (and no longer wakes the LLM adjudicator for a purely human pair).

Test verification (RED → GREEN)

Two database-backed tests (they skip without UTOPIA_DATABASE_URL, like the rest of the utopia-store suite; the CI migrations job runs them with UTOPIA_TEST_REQUIRE_DB=1).

RED — on the unmodified dev branch, the store test fails because the mention silently attaches:

thread 'a_namesake_tie_creates_an_entity_and_two_reviews' panicked at
crates/utopia-store/tests/a_namesake_tie_goes_to_review_not_a_coin_flip.rs:
同名并列不该静默归并到先遇到的那个——该新建实体(#270)
test result: FAILED. 0 passed; 1 failed

GREEN — with the fix, a third entity is created and two human review pairs are filed:

test a_namesake_tie_creates_an_entity_and_two_reviews ... ok            (utopia-store)
test extraction::tests::a_profile_tie_across_documents_files_human_reviews_not_an_attach ... ok   (utopia-server)
test result: ok. 316 passed; 0 failed        (cargo test --workspace, against Postgres)

The server test drives the real resolve → resolve_uncached path and asserts the tie files two human reviews, leaves pending_adjudications empty, and never sets needs_adjudication.

Local CI

Ran the full ci.yml locally against pgvector/pgvector:pg16:

  • cargo fmt --all --check — clean
  • cargo clippy --workspace --all-targets -- -D warnings — clean
  • cargo test --workspace (with a database) — 316 passed, 0 failed
  • cargo build --workspace — ok
  • web: pnpm install --frozen-lockfile && pnpm build — ok

When a base holds two same-name entities with identical profile vectors
(common after deeplethe#221 seeds both from one chunk), a later bare mention scores
the same against both. resolve_mention attached to the first candidate the
database returned once the score cleared SIM_ATTACH, filing no review — so
which entity got the fact was decided by candidate order.

A near-tie among namesakes (same canonical name, scores within the new
SIM_TIE_MARGIN) is now a grey zone regardless of the absolute score: attach
to neither, create the entity, and file a review pair against each candidate
at the human stage, so the batch adjudicator can never auto-merge two
profiles it cannot tell apart. SIM_ATTACH / SIM_NEW are untouched.

Fixes deeplethe#270.

Signed-off-by: Ben Younes <2910651+ousamabenyounes@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: WaylandYang <wayland0916@gmail.com>
@WaylandYang

Copy link
Copy Markdown
Contributor

Reviewed and pushed one commit on top: the new namesake_tie reason had no wording in i18n, so escalationText would have fallen through and rendered the raw namesake_tie|0.61 on the card — the same gap #310 closed for namesake last week. English and Chinese added; the similarity figure stays visible here, since 0.61 is a real profile score rather than the sentinel that made #310 hide it.

The rest reads well. Two things I checked and liked: the reduce keeps first-on-tie, so the old sim > s ordering is preserved rather than quietly changed, and stage being a struct field means the four existing call sites had to state their intent — a missed one would not compile. The RED output on unmodified dev is what makes the test worth having.

Approving CI now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Two namesakes with identical profiles attach a later mention by candidate order

2 participants