Skip to content

A row a rule implies is attested by the statements it read, not by the moment it was computed - #1012

Merged
WaylandYang merged 3 commits into
deeplethe:devfrom
Maya-Kid:fix/implied-rows-follow-their-statements
Sep 30, 2026
Merged

WaylandYang merged 3 commits into
deeplethe:devfrom
Maya-Kid:fix/implied-rows-follow-their-statements

Conversation

@Maya-Kid

Copy link
Copy Markdown
Contributor

Follows #995. A row that an approved rule implies (0073) takes the moment it was computed as its attestation. Since #995, a fact with no time holds at every moment (0064 decision 5), and a row materialised from a statement follows that statement's attestation. A rule's row does neither.

Problem

  • Insert. imply_in_tx writes a rule's row through insert_fact_on, which stores COALESCE(attested_at, now()). That fallback is meant for a fact a person writes.
    • A phrase rule passes its statement's attestation, so an unattested statement gives the processing moment.
    • A kind-word rule passes NULL, with the same result.
  • Sync. sync_typed_attestation puts a typed row back on its statements' attestation, but it has two gaps:
    • It reads only typed_fact_sources.
    • In materialize_in_tx it runs before imply_in_tx.
  • Later attestation. A statement's attestation can change after its rows are computed, and a rule's row does not follow it. attested_by also stays empty. This happens two ways:
    • Time resolution attests a document's statements in its own job, then calls the sync.
    • An earlier document can say the same thing again; attest_statement then moves the attestation earlier.
  • So two undated statements of the same shape read differently. Measured on dev, one statement was under a bound phrase and one under an approved rule:
row attested_from holds in 2000
materialised by the binding null yes
implied by the rule the moment materialisation ran no

Change

  • In crates/utopia-store/src/materialize.rs:
    • Sources. sync_typed_attestation takes a row's source statements from both typed_fact_sources and implied_fact_sources. A kind-word rule's source is an entity, so it adds no attestation.
    • Value. Unchanged: the earliest attestation among the sources, with its date's name, or none.
    • Rows moved. The rows it moved before (those with a statement materialised into them), plus the rows a rule computed (implied).
    • Not moved. A row a person wrote, which only a rule's conclusion merged into, stays as it is today. attest_earlier already moved it earlier at the merge.
    • Order. In materialize_in_tx the sync now runs after imply_in_tx.
  • migrations/0101_a_row_a_rule_implies_follows_its_statements.sql applies the same rule to rows written before. Like 0100, it covers every base and invalidated rows too. CURRENT_SCHEMA_VERSION becomes 92.
  • 0064 decision 5 now says that a rule's row follows the statements the rule read, and names 0101.
  • The comments at the two callers of the sync (graph::attest_statement and time_resolution::attest_statements) now mention rule rows.

#1007 also adds a 0101. Whichever lands second moves to 0102 and a schema version of 93.

Verification

Run in containers on a Linux host: Rust 1.98.1 and PostgreSQL 16 with pgvector.

  • a_row_a_rule_implies_follows_the_statement_it_read (store tests):
    • Undated statement. The rule's row and a kind-word row have no attestation, and both hold in 2015 and now.
    • Statement attested afterwards, the way time resolution does it (attest_statement, then sync_typed_attestation):
      • the rule's row moves to that day, with the date's name, and does not hold before it;
      • the kind-word row stays unattested;
      • running materialisation again changes neither row.
    • A person's row that the rule's conclusion merged into keeps the moment it was written.
  • a_row_a_rule_implied_follows_its_statements is a top-level binary, like the 0096 and 0099 tests, because the migration is not filtered by base.
    • It sets the rule rows back to the processing moment, as dev writes them, and then runs 0101 twice.
    • Undated statement: null.
    • Statement attested after materialisation: that day, with the date's name.
    • Kind word: null.
    • The person's row: unchanged.
  • Control. On dev with only the two tests, plus a 0101 that does nothing so that the second test compiles, both tests fail:
    • The store test fails at its first check. The rule's row is attested at the moment materialisation ran (2026-09-30T04:09:11Z), not left empty.
    • The migration test fails at its first check after 0101 runs. The undated statement's row keeps the processing moment.
  • Migration on dev's own rows. 0101 was also applied by hand to rows written by dev's code.
    • Three rule rows were at the processing moment. They became null, 2026-09-04 (签约日 2026年9月4日), and null.
    • The person's row did not change.
  • cargo fmt --all --check and cargo clippy --locked --workspace --all-targets -- -D warnings are clean.
  • UTOPIA_TEST_REQUIRE_PDFTOTEXT=1 cargo test --locked --workspace passes: 1306 passed, 0 failed.

…e moment it was computed

Signed-off-by: dada-yan <BinjunYann@gmail.com>

@WaylandYang WaylandYang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. Thanks @Maya-Kid.

Reviewed at 1c599d9. The change closes the gap the table in the description shows: a rule's row now reads its statements' attestation through implied_fact_sources, the sync runs after imply_in_tx, and a person's row that a conclusion merged into is left alone because it is neither implied nor materialised from a statement. The migration follows 0100's shape (every base, invalidated rows too). The test that attests a statement afterwards, the way time resolution does, is the one that pins the behaviour that mattered.

Migration number. 0101 is also taken by #1015 (the Ark OCR reader), which is waiting on a contributor's verification with a real key. This one lands first and keeps 0101; #1015 will move to 0102 and a schema version of 93.

@WaylandYang
WaylandYang merged commit fe47a8b into deeplethe:dev Sep 30, 2026
6 checks passed
WaylandYang added a commit that referenced this pull request Sep 30, 2026
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Wayland Yang <wayland0916@gmail.com>
WaylandYang added a commit that referenced this pull request Sep 30, 2026
…e Ark vision model read in one pass (#1015)

* The OCR reader chooses its provider: MinerU as before, or a Volcengine Ark vision model read in one pass

Scans and images can now be read by an Ark vision model beside the
MinerU service. llm_settings gains ocr_provider (default mineru) and
ocr_model; a key stays with its provider and is cleared when the
provider changes, in one SQL statement. The Ark path sends an image
whole, renders a PDF page by page with Poppler, asks the model for the
written text of each page as JSON, retries a page that hit a rate limit,
a timeout or a server error three times inside the pass, and builds the
same Reading as MinerU. Nothing is remembered between passes: a failed
pass re-reads every page on the document's ordinary retry. ADR 0065.

The prompt, the probe image, the URL rules and the Poppler subprocess
handling come from Fonna's implementation in #1007, which this narrows.

Co-authored-by: Fonna <jiangpeng274@outlook.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Wayland Yang <wayland0916@gmail.com>

* The Ark OCR migration moves to 0102: 0101 landed with #1012

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Wayland Yang <wayland0916@gmail.com>

* 补齐方舟 OCR 的响应上限、配置兼容与验收 (#1016)

* 修复方舟 OCR 的响应读取上限与旧客户端配置兼容

在读取响应时执行大小上限,复用现有代理规则。省略供应商的旧请求保留已有配置,显式切换仍隔离密钥。补充整篇重读、空白页页码和真实接口验收,并更新 Agent Plan 地址提示与设计说明。

Signed-off-by: Fonna <jiangpeng274@outlook.com>

* 补强扫描原文验收并处理测试替身的部分读取

Signed-off-by: Fonna <jiangpeng274@outlook.com>

---------

Signed-off-by: Fonna <jiangpeng274@outlook.com>

* The README says which readers scans, images and recordings go to

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Wayland Yang <wayland0916@gmail.com>

---------

Signed-off-by: Wayland Yang <wayland0916@gmail.com>
Signed-off-by: Fonna <jiangpeng274@outlook.com>
Co-authored-by: Fonna <jiangpeng274@outlook.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Fonna <35397279+Fonna@users.noreply.github.com>
@Maya-Kid
Maya-Kid deleted the fix/implied-rows-follow-their-statements branch September 30, 2026 11:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants