Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,13 @@ Opening a cutoff-rewritten title shows **Body this run knew** from
`source_post_revision` beside the live rewrite (ADR 0025 / v2.1.0).
Do not invent the earlier sentence when no revision covers the cutoff.

A corporate-entity similarity result has three outcomes: unique, miss,
or tie (ADR 0026). A tie is not a miss. Keep the organization name
unbound and do not create an `AUTO-` catalog row, even when live name
resolution, hierarchy inference, and verification are available. Keyman
must test the raw organization name before any abbreviation rewrite so a
rewrite cannot turn an existing tie into an apparent creation miss.

## CI gates

`.github/workflows/tests.yml` runs the full suite on every PR to `main`.
Expand Down
6 changes: 6 additions & 0 deletions CHANGELOG.d/tied-organization-no-auto-create.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Tied organization names do not create catalog rows

A tied top organization similarity score now stays unbound. Even with live
name resolution, hierarchy inference, and verification, the ingestion path
does not insert an `AUTO-` catalog row. Only a genuine below-threshold miss
may enter the corroborated creation path (ADR 0026).
45 changes: 28 additions & 17 deletions backend/app/corporate_entity_ingestion.py
Original file line number Diff line number Diff line change
@@ -1,14 +1,15 @@

"""Resolve an organization mention to the corporate hierarchy catalog.

Existing similarity matches are reused. A previously unseen entity is
created only after inference proposes its complete hierarchy placement
and external verification corroborates that placement. Parent failure,
cycles, and excessive depth all fail closed. See ADR 0010.
Existing unique similarity matches are reused. A tied top score stays
unbound and does not create a row (ADR 0026). A previously unseen entity
-- no candidate at or above the similarity threshold -- is created only
after inference proposes its complete hierarchy placement and external
verification corroborates that placement. Parent failure, cycles, and
excessive depth all fail closed. See ADR 0010.

Creation writes take one named Postgres advisory transaction lock
(``pg_advisory_xact_lock``) after network inference/verification, then
reload catalog candidates before inserting. See ADR 0012.
reload catalog candidates before inserting. See ADR 0012.
"""

from __future__ import annotations
Expand All @@ -23,8 +24,10 @@
HierarchyProposal,
)
from lineageweave.corporate_hierarchy_resolution import (
RESOLUTION_TIE,
RESOLUTION_UNIQUE,
CorporateEntityCandidate,
resolve_corporate_entity,
score_corporate_entity,
)
from lineageweave.relation_verification import (
STATUS_CORROBORATED,
Expand Down Expand Up @@ -112,9 +115,13 @@ async def get_or_create_corporate_entity(
) -> str | None:
"""Return a verified catalog id, otherwise ``None``.

A proposed parent must independently corroborate and resolve before
the child can be inserted. Repeated names in the recursion path are
cycles, including multi-node cycles such as A -> B -> A.
A unique similarity match is reused. A tied top score stays unbound
and does not create a third same-named row (ADR 0026). Only a genuine
miss -- no candidate at or above ``min_similarity`` -- may enter ADR
0010 inference. A proposed parent must independently corroborate and
resolve before the child can be inserted. Repeated names in the
recursion path are cycles, including multi-node cycles such as
A -> B -> A.
"""
normalized_name = organization_name.strip()
if not normalized_name:
Expand All @@ -123,9 +130,11 @@ async def get_or_create_corporate_entity(
if visit_key in _visited_names:
return None

existing_id = resolve_corporate_entity(normalized_name, candidates)
if existing_id is not None:
return existing_id
existing = score_corporate_entity(normalized_name, candidates)
if existing.kind == RESOLUTION_UNIQUE and existing.catalog_id is not None:
return existing.catalog_id
if existing.kind == RESOLUTION_TIE:
return None
Comment thread
seonghobae marked this conversation as resolved.
if _depth >= _MAX_HIERARCHY_DEPTH or not inference_client.available:
return None

Expand Down Expand Up @@ -176,13 +185,15 @@ async def get_or_create_corporate_entity(
"select pg_advisory_xact_lock(hashtext($1))",
_CREATION_LOCK_KEY,
)
fresh_existing_id = resolve_corporate_entity(
fresh = score_corporate_entity(
normalized_name,
await _reload_candidates(conn),
)
if fresh_existing_id is not None:
_remember_candidate(candidates, fresh_existing_id, normalized_name)
return fresh_existing_id
if fresh.kind == RESOLUTION_UNIQUE and fresh.catalog_id is not None:
_remember_candidate(candidates, fresh.catalog_id, normalized_name)
return fresh.catalog_id
if fresh.kind == RESOLUTION_TIE:
return None
new_id = await _create_entity(
conn,
normalized_name,
Expand Down
69 changes: 52 additions & 17 deletions backend/app/keyman_ingestion.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,11 @@
search-corroborated hierarchy placement (level + parent) before
creating a real new row, so the "통합 고객사 계열 tree AI" requirement
is actually populated from real extraction, not left permanently empty.

Tie boundary (ADR 0026): a raw organization name whose distinct catalog
candidates share the top qualifying similarity score stays unbound before
abbreviation rewriting. Live name resolution therefore cannot turn known
ambiguity into an apparent miss and manufacture a third `AUTO-` row.
"""

from __future__ import annotations
Expand All @@ -52,7 +57,11 @@
CorporateHierarchyInferenceClient,
NullCorporateHierarchyInferenceClient,
)
from lineageweave.corporate_hierarchy_resolution import CorporateEntityCandidate
from lineageweave.corporate_hierarchy_resolution import (
RESOLUTION_TIE,
CorporateEntityCandidate,
score_corporate_entity,
)
from lineageweave.keyman_extraction import KeymanExtractionClient, PersonMention
from lineageweave.organization_name_resolution import (
NullOrganizationNameResolutionClient,
Expand Down Expand Up @@ -113,7 +122,6 @@ async def _upsert_person(conn: asyncpg.Connection, mention: PersonMention) -> st
return str(row["person_id"])



async def _upsert_affiliation(
conn: asyncpg.Connection,
person_id: str,
Expand Down Expand Up @@ -166,6 +174,38 @@ async def _upsert_affiliation(
)


async def _resolve_affiliated_organization(
conn: asyncpg.Connection,
organization_name: str,
context_text: str,
resolution_client: OrganizationNameResolutionClient,
verification_client: RelationVerificationClient,
hierarchy_inference_client: CorporateHierarchyInferenceClient,
candidates: list[CorporateEntityCandidate],
) -> tuple[str, str, str | None]:
"""Resolve one affiliation without rewriting a known raw-name tie."""
raw_outcome = score_corporate_entity(organization_name, candidates)
if raw_outcome.kind == RESOLUTION_TIE:
return organization_name, organization_name, None

resolved_name = await resolve_organization_name(
conn,
resolution_client,
verification_client,
organization_name,
context_text,
)
corporate_entity_id = await get_or_create_corporate_entity(
conn,
resolved_name,
context_text,
hierarchy_inference_client,
verification_client,
candidates,
)
return organization_name, resolved_name, corporate_entity_id


async def ingest_post_keymen(
conn: asyncpg.Connection,
client: KeymanExtractionClient,
Expand Down Expand Up @@ -206,22 +246,17 @@ async def ingest_post_keymen(
for mention in mentions:
resolved_orgs: list[tuple[str, str, str | None]] = []
for organization_name in mention.affiliated_organization_names:
resolved_name = await resolve_organization_name(
conn,
resolution_client,
verification_client,
organization_name,
post_body,
)
corporate_entity_id = await get_or_create_corporate_entity(
conn,
resolved_name,
post_body,
hierarchy_inference_client,
verification_client,
candidates,
resolved_orgs.append(
await _resolve_affiliated_organization(
conn,
organization_name,
post_body,
resolution_client,
verification_client,
hierarchy_inference_client,
candidates,
)
)
resolved_orgs.append((organization_name, resolved_name, corporate_entity_id))
resolved_by_mention.append((mention, resolved_orgs))

normalized_mentions: list[PersonMention] = []
Expand Down
97 changes: 97 additions & 0 deletions docs/adr/0026-tied-organization-similarity.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
# ADR 0026 — Tied organization similarity stays unbound

**Decision status:** Accepted
**Date:** 2026-08-17

## Context

Role-and-responsibility and Keyman ingestion resolve free-text organization
names against `corporate_entity`. The former resolver returned either one
catalog id or `None`. That collapsed two materially different outcomes:

1. no candidate met the minimum similarity threshold; and
2. two or more distinct catalog ids shared the best score.

A genuine miss may enter ADR 0010's inference-and-corroboration path. A tie
must not. Treating a tie as a miss can create a third, deterministic
`AUTO-...` row for a name that is already represented by multiple catalog
records. It can also bind whichever homonym happened to appear first in an
unordered candidate result.

Keyman adds another boundary: it may run a verified abbreviation rewrite
before hierarchy resolution. If a raw tied name is rewritten first, the new
string can appear to be a miss and incorrectly enter the creation path.

Fellegi and Sunter's record-linkage decision framework retains an uncertain
region rather than forcing a match. In this product, an equal top score is
that review state. String similarity remains candidate generation, not proof
of identity.

## Decision

`score_corporate_entity` classifies each organization mention as:

- `unique`: exactly one distinct catalog id has the top score at or above
the threshold;
- `miss`: no candidate reaches the threshold; or
- `tie`: multiple distinct catalog ids share the top qualifying score.

Only `unique` returns a catalog id. Only `miss` may continue into ADR 0010
inference and corroborated creation. `tie` returns unbound immediately.

The same classification is repeated after the advisory creation lock and
catalog reload. If concurrent writes make the refreshed result a tie, no
insert occurs.

Keyman evaluates the raw organization name before abbreviation rewriting.
A raw tie bypasses name resolution and hierarchy inference, remains text,
and stores no new catalog id. This prevents a resolver rewrite from turning
known ambiguity into an apparent miss.

Duplicate candidate rows carrying the same `corporate_entity_id` are one
candidate, not a tie.

```mermaid
flowchart TD
mention[Organization mention] --> raw[Score raw catalog candidates]
raw --> outcome{Resolution outcome}
outcome -->|unique| bind[Bind unique catalog id]
outcome -->|tie| hold[Keep unbound; no AUTO row]
outcome -->|miss| enrich[Optional verified name resolution]
enrich --> score[Score resolved name]
score --> resolved{Resolution outcome}
resolved -->|unique| bind
resolved -->|tie| hold
resolved -->|miss| create[ADR 0010 infer and corroborate]
create --> lock[Lock and reload candidates]
lock --> refreshed{Refreshed outcome}
refreshed -->|unique| bind
refreshed -->|tie| hold
refreshed -->|miss| insert[Insert AUTO row]
```

## Consequences

- Equal top scores are deterministic and fail closed rather than depending
on row order.
- A tied organization name never creates an `AUTO-` catalog row, including
with live resolver, inference, and verification clients.
- Genuine misses retain the existing, corroborated hierarchy creation path.
- Buyers see ambiguous organization names as text until the catalog has a
unique identity decision.
- Future collective entity resolution may use relational context to resolve
ties, but must publish a reviewed unique result before binding.

## References — APA 7th

Bhattacharya, I., & Getoor, L. (2007). Collective entity resolution in
relational data. *ACM Transactions on Knowledge Discovery from Data, 1*(1),
Article 5. https://doi.org/10.1145/1217299.1217304

Christen, P. (2012). *Data matching: Concepts and techniques for record
linkage, entity resolution, and duplicate detection*. Springer.
https://doi.org/10.1007/978-3-642-31164-2

Fellegi, I. P., & Sunter, A. B. (1969). A theory for record linkage.
*Journal of the American Statistical Association, 64*(328), 1183–1210.
https://doi.org/10.2307/2286061
Loading