Skip to content

A trimmed description is not stale - #673

Merged
WaylandYang merged 1 commit into
devfrom
fix/a-trimmed-description-is-not-stale
Sep 13, 2026
Merged

WaylandYang merged 1 commit into
devfrom
fix/a-trimmed-description-is-not-stale

Conversation

@WaylandYang

Copy link
Copy Markdown
Contributor

Fixes #672.

What was wrong

Whether an ontology row's vector is stale is decided by comparing the text it was embedded from (embedded_text) with the text it would be embedded from now. The stored side comes from embed_text, which uses Rust str::trim. The comparison side was rebuilt in SQL with btrim(label) / btrim(description), which without a character argument strips only ASCII spaces.

Descriptions that begin or end with a newline or tab, which is how schema.org descriptions come in ("...\n "), therefore never matched. Those rows stayed stale forever: embed_ontology re-embedded them on every refresh, and since #640 the extraction gate (gate_required) deferred every extract_document until DEFER_WINDOW_SECS ran out and the document failed. On the recall bench (recall-bench schema-org) that was 59 rows, and 0 of 64 chunks extracted in 15 minutes.

What changed

types_needing_embedding and relations_needing_embedding load each row's label, description and stored embedded / embedded_model / embedded_text (plus the label-vector equivalents for entity types) and decide staleness in Rust with the same embed_text that produces the text. This is the approach mappings::needing_embedding already takes, for the same reason.

Verification

New DB-backed test a_trimmed_description_is_not_stale. An entity type with a label wrapped in tabs and a description ending in "\n ", and a relation whose description starts and ends with newlines:

  • the first pass embeds all three (entity full, entity label, relation full), and the label text is sent trimmed;
  • after set_type_embeddings, nothing is stale;
  • changing a description makes exactly that row stale again;
  • switching the model makes all three stale.

Against the previous code the test fails with all three rows reported stale right after being embedded. End-to-end results from the recall bench follow in a comment.

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: WaylandYang <wayland0916@gmail.com>
@WaylandYang

Copy link
Copy Markdown
Contributor Author

End to end on the recall bench (utopia_edges copy, KB recall-bench schema-org, SiliconFlow):

build chunks extracted score
dev (with #666, before this fix), two rounds 0 / 64 in 15 min: every extract_document deferred, embed_ontology re-embedding the same 59 rows 0 / 52
dev + this fix 64 / 64 in about 26 min 42 / 52 (relations 9/12, literals 32/39)

42/52 is within one σ (≈ 2.7) of the 09-11 dev baselines (40–41/52), so nothing merged into dev today, #666's prompt change included, shows as a regression on this bench.

@WaylandYang
WaylandYang merged commit c927c90 into dev Sep 13, 2026
@WaylandYang
WaylandYang deleted the fix/a-trimmed-description-is-not-stale branch September 13, 2026 12:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Extraction waits forever when an ontology description ends in a newline

1 participant