Repository navigation
A corrected fact keeps the statements it was computed from - #967
Conversation
Signed-off-by: dada-yan <BinjunYann@gmail.com>
WaylandYang
left a comment
There was a problem hiding this comment.
Thank you. This is the third rewriter #911 missed, fixed the same way as the other two, and the test pins all four consequences: the links, the three readings of the graph, the idle next materialization, and the single row. The bench numbers before and after are what makes it easy to accept. Landing it.
On rows corrected before this change: yes, please, as a follow-up PR with its own migration. Restore from_statement_id, implied and the source rows on a live typed row whose supersedes chain leads to a materialized row, and retire the duplicate that a later materialization computed from the same statement by invalidating it, never deleting it; the ledger stays append-only. Have the migration RAISE NOTICE the two counts so an operator sees what it did. Use plain statements, no CONCURRENTLY, and count the files in migrations/ for the CLI constant when you rebase. Thanks again.
WaylandYang
left a comment
There was a problem hiding this comment.
Maintainer review: sound, landing.
…cates are retired (#972) Signed-off-by: dada-yan <BinjunYann@gmail.com> Co-authored-by: WaylandYang <145302500+WaylandYang@users.noreply.github.com>
Problem
After a person corrects the dates of a materialized fact, the statement the fact was computed from comes back as its own edge, with its old dates. The next materialization then computes a second typed row from that statement.
The temporal bench shows it on dev (f1a36a5), in the corpus's own correction step. The corpus's wave 3 corrects Zhang San's start on Project Aurora from 2023-01-10 to 2023-02-01 ("The charter date was the approval date"). Afterwards the ledger holds:
from_statement_idand notyped_fact_sourcesThe open statement is no longer represented by a live typed row, so the canvas, the entity panel and the chat tools draw it again: Zhang San leads Aurora from 2023-01-10 with no end.
Why
REPRESENTED_BY_TYPEDingraph.rs).rewrite_end_tx) and rehoming (rehome_tx), carryfrom_statement_id,implied,typed_fact_sourcesandimplied_fact_sourcesonto the corrected row, so the next round sees it as already computed.correct_interval, behindPATCH /kbs/{id}/facts/{fact_id}) is a third rewrite, and it still writes the corrected row without them.Change
correct_intervalcarriesfrom_statement_idandimpliedin its insert and callscopy_materialization_links, as the other two rewrites do.end_derivedstays false, since the person set both ends.Verification
Run in containers on a Linux host: Rust 1.98.1 and Postgres 16 with pgvector.
New test
a_corrected_time_keeps_its_statements(store) goes through the same path as the handler:leadsare materialized. Li Si's handover closes Zhang San's row.correct_intervalmoves Zhang San's start to 2023-02-01, andreconcile_moved_factsruns.It then checks four things:
from_statement_idand as its only source.graph::neighborhoodshows Aurora's leads as none on 2023-01-20, Zhang San on 2023-03-01 and Li Si on 2024-08-01.materializedoes nothing.On dev (f1a36a5) with only the new test
(None, [])where the charter statement was expected.[]materializeThis branch
cargo fmt --all --checkis clean.cargo clippy --locked --workspace --all-targets -- -D warningsis clean.UTOPIA_TEST_REQUIRE_PDFTOTEXT=1 cargo test --locked --workspacepasses, including all store tests.Temporal bench
The bench was run on a fresh ingestion, with DeepSeek (
deepseek-chat) for chat and extraction and bge-m3 for embeddings.works_foras an interval that holds at no moment, which is A statement dated at one moment, bound to a state property, becomes a state that holds at no moment #966.Rows corrected before this change
This change fixes new corrections only.
Repairing them takes a migration:
supersedeschain leads to a materialized row, setfrom_statement_idandimpliedfrom that row and restore the sources.rc7 includes #911 and this path, so databases on it may hold such rows. I left the migration out because it is a separate decision: whether you want it, and whether it should retire duplicates automatically or only report them. I can add it here or in a follow-up.