Skip to content

A corrected fact keeps the statements it was computed from - #967

Merged
WaylandYang merged 1 commit into
deeplethe:devfrom
Maya-Kid:fix/corrected-time-keeps-statements
Sep 27, 2026
Merged

WaylandYang merged 1 commit into
deeplethe:devfrom
Maya-Kid:fix/corrected-time-keeps-statements

Conversation

@Maya-Kid

Copy link
Copy Markdown
Contributor

Problem

After a person corrects the dates of a materialized fact, the statement the fact was computed from comes back as its own edge, with its old dates. The next materialization then computes a second typed row from that statement.

The temporal bench shows it on dev (f1a36a5), in the corpus's own correction step. The corpus's wave 3 corrects Zhang San's start on Project Aurora from 2023-01-10 to 2023-02-01 ("The charter date was the approval date"). Afterwards the ledger holds:

fact layer from → to live
Zhang San leads Project Aurora open (the charter's statement) 2023-01-10 → (open) yes
Zhang San leads Project Aurora typed, closed by Li Si's handover 2023-01-10 → 2024-07-05 no, corrected
Zhang San leads Project Aurora typed, the correction 2023-02-01 → 2024-07-05 yes, but no from_statement_id and no typed_fact_sources

The open statement is no longer represented by a live typed row, so the canvas, the entity panel and the chat tools draw it again: Zhang San leads Aurora from 2023-01-10 with no end.

  • Ledger. Five of the bench's questions fail on it: 29/37 on dev. The graph shows Zhang San leading Aurora on 2023-01-20, before the corrected start. It also shows him leading in August 2024, June 2025, October 2025 and January 2026, after his tenure was closed.
  • Chat answers from the same graph:
    • "In August 2024, Project Aurora had two people recorded as leading it", listing Zhang San "leading since 10 January 2023 (ongoing)" and Li Si.
    • "In January 2026, Project Aurora is led by both Zhang San (since 2023-01-10) and Zhou Qi (since 2025-09-01)."
    • "On 20 January 2023, Project Aurora was led by Zhang San — he took over as lead on 2023-01-10."

Why

Change

  • correct_interval carries from_statement_id and implied in its insert and calls copy_materialization_links, as the other two rewrites do.
  • The comment on the function says why: a person corrects the time, not where the fact came from.
  • The rest is unchanged:
    • end_derived stays false, since the person set both ends.
    • Open conflicts are still not carried; the handler reconciles right after, so a collision the new interval still has is recorded again.

Verification

Run in containers on a Linux host: Rust 1.98.1 and Postgres 16 with pgvector.

New test

a_corrected_time_keeps_its_statements (store) goes through the same path as the handler:

  • Two open statements bound to leads are materialized. Li Si's handover closes Zhang San's row.
  • correct_interval moves Zhang San's start to 2023-02-01, and reconcile_moved_facts runs.

It then checks four things:

  • The corrected row still has the charter's statement as from_statement_id and as its only source.
  • graph::neighborhood shows Aurora's leads as none on 2023-01-20, Zhang San on 2023-03-01 and Li Si on 2024-08-01.
  • The next materialize does nothing.
  • Zhang San has one typed row, 2023-02-01 → 2024-07-05.

On dev (f1a36a5) with only the new test

  • The test fails at its first check: (None, []) where the charter statement was expected.
  • A copy that prints each check instead of asserting shows the rest:
check dev this branch
corrected row's statement and sources none, [] the charter statement
Aurora's leads on 2023-01-20 Zhang San none
on 2023-03-01 Zhang San, Zhang San Zhang San
on 2024-08-01 Li Si, Zhang San Li Si
next materialize adds 1 row, closes 1 nothing
Zhang San's typed rows afterwards 2023-02-01 → 2024-07-05 and 2023-01-10 → 2024-07-05 2023-02-01 → 2024-07-05

This branch

  • The new test passes.
  • cargo fmt --all --check is clean.
  • cargo clippy --locked --workspace --all-targets -- -D warnings is clean.
  • UTOPIA_TEST_REQUIRE_PDFTOTEXT=1 cargo test --locked --workspace passes, including all store tests.

Temporal bench

The bench was run on a fresh ingestion, with DeepSeek (deepseek-chat) for chat and extraction and bge-m3 for embeddings.

ledger world axis record axis
dev f1a36a5 29/37 15/23 14/14
this branch 34/37 20/23 14/14
  • The five Aurora questions now pass.
  • The three that still fail are not this bug:
  • Chat, on the seven Aurora questions. On dev, three answers were wrong: two leads at once, twice, and Zhang San leading on 2023-01-20. On this branch all seven name the right lead, or say no one led it, for example "On 20 January 2023, no one is recorded as leading Project Aurora: the earliest leads fact starts on 1 February 2023". Most of them still count as failures under the bench's substring scoring, because they mention the predecessor. That scoring limit is the one the README already describes.

Rows corrected before this change

This change fixes new corrections only.

  • A row corrected on an earlier build still has no sources.
  • If a materialization has run since that correction, the statement already has a second typed row with the old interval.

Repairing them takes a migration:

  • For a live typed row whose supersedes chain leads to a materialized row, set from_statement_id and implied from that row and restore the sources.
  • Retire the duplicate that materialization computed afterwards.

rc7 includes #911 and this path, so databases on it may hold such rows. I left the migration out because it is a separate decision: whether you want it, and whether it should retire duplicates automatically or only report them. I can add it here or in a follow-up.

Signed-off-by: dada-yan <BinjunYann@gmail.com>

@WaylandYang WaylandYang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you. This is the third rewriter #911 missed, fixed the same way as the other two, and the test pins all four consequences: the links, the three readings of the graph, the idle next materialization, and the single row. The bench numbers before and after are what makes it easy to accept. Landing it.

On rows corrected before this change: yes, please, as a follow-up PR with its own migration. Restore from_statement_id, implied and the source rows on a live typed row whose supersedes chain leads to a materialized row, and retire the duplicate that a later materialization computed from the same statement by invalidating it, never deleting it; the ledger stays append-only. Have the migration RAISE NOTICE the two counts so an operator sees what it did. Use plain statements, no CONCURRENTLY, and count the files in migrations/ for the CLI constant when you rebase. Thanks again.

@WaylandYang WaylandYang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maintainer review: sound, landing.

@WaylandYang
WaylandYang merged commit ab66bd7 into deeplethe:dev Sep 27, 2026
WaylandYang added a commit that referenced this pull request Sep 27, 2026
…cates are retired (#972)

Signed-off-by: dada-yan <BinjunYann@gmail.com>
Co-authored-by: WaylandYang <145302500+WaylandYang@users.noreply.github.com>
@Maya-Kid
Maya-Kid deleted the fix/corrected-time-keeps-statements branch September 28, 2026 09:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants